arXiv Papers with Code in Computer Science (July 2026)
Authors:Yixiao Wang, Cheng-En Wu, Lingfeng Sun, Pengcheng Wang, Xiang Ji, Boyuan Liang, Guojian Zhan, Masayoshi Tomizuka
Abstract:
Sequential robot manipulation requires policies to execute novel combinations of familiar instruction components. However, collecting demonstrations for all possible instruction tuples is combinatorially expensive, while sparsely covered datasets often fail under out‑of‑distribution recombination. This paper studies compositional generalization through the lens of instruction‑space coverage. We decompose the generalization gap into three sources: marginal instruction shift, instruction‑compositional shift, and context‑‑action shift. This decomposition allows us to diagnose when sparse training coverage is sufficient, and what structure the training set must preserve for reliable action prediction. Our results show that exhaustive tuple enumeration is unnecessary: a structured subset, as small as one quarter of the full task space, can recover strong out‑of‑distribution performance when it covers action‑relevant dependencies. We further find that sparse training often fails due to instruction steering rather than missing low‑level skills; finetuning only one demonstration per task improves OOD success from \(0.4%\) to \(54.7%\). For semantically dependent tasks, effective coverage must capture relational structure rather than only factor diversity. These findings suggest that efficient robot data collection should prioritize dependency coverage in instruction space over exhaustive task expansion. More results are available in the supplementary material. Project website: https://yixiaowang7.github.io/Diagnosing_Compositional_Generalization_Robot_Page/.
Authors:Muyao Niu, Mingze Ma, Yifan Zhan, Qingtian Zhu, Zhihang Zhong, Wei Guo, Chang Wen Chen, Yinqiang Zheng
Abstract:
Robust low‑light imaging remains challenging for the community. Recent studies have explored fusing Near‑Infrared (NIR) with noisy RGB to achieve improved enhancement, yet most methods depend on carefully curated training data pairs, with limited robustness under different scenarios. This paper offers a new perspective for RGB‑NIR low‑light imaging by incorporating 3D‑aware neural modeling. Without using clean RGB supervision, a powerful model can be optimized to implicitly fuse extremely noisy RGB observations with NIR cues in 3D space, effectively recovering clean RGB images. The proposed model obviates the requirement for clean RGB data collection, generalizes across different noise levels. Extensive evaluations on synthetic and real data demonstrate its superiority. Codes available: https://github.com/MyNiuuu/3DarkFusion
Authors:Zilong Chen, Chaorui Deng, Kunchang Li, Hongyi Yuan, Haoqi Fan
Abstract:
We study empirical scaling properties for text conditioning in visual generation. Such properties have rarely been measured because diffusion loss does not scale with the number of tokens in natural‑language prompts. Surprisingly, we find that the converged diffusion loss scales with the amount of structured language in the prompt. To quantify structured language, we adapt two complementary measures: a white‑box likelihood metric (GPG) and a black‑box attribute metric (ED). Across controlled training runs, the converged diffusion loss decreases approximately linearly with GPG and follows a power law with ED. Guided by these scaling properties, we improve \emphdiffusability by constructing structured prompts with semantic and geometric annotations derived from images, and improve \emphpromptability by training a prompter through supervised fine‑tuning, cold‑start, and verifier‑gated on‑policy distillation. The resulting system outperforms all evaluated open‑weight models on nearly every compositional, reasoning, and world‑knowledge benchmark, while matching or surpassing the strongest closed‑weight models on most evaluations.
Authors:Boyang Zhang, Adrian Lyjak, Eli Stewart, Zhaoqi Li, Simon Suo
Abstract:
Enterprise workflows increasingly rely on agents for \emphschema‑guided extraction: given a document and a user‑defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema‑guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground‑truth curation pipeline combines independent‑system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order‑insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word‑ and page‑level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost. Dataset and evaluation code are available on \hrefhttps://huggingface.co/datasets/llamaindex/ExtractBenchHuggingFace and \hrefhttps://github.com/run‑llama/ExtractBenchGitHub.
Authors:Maria Smirnova, Alexey Kravatskiy
Abstract:
SignMuon compresses the Muon update to one bit per parameter by taking its elementwise sign, providing the most direct way to run a matrix‑aware optimizer under an extremely low communication budget. It outperforms SignSGD in practice, yet it can ascend even on a linear function. Signing the gradient before the Linear Minimization Oracle (LMO), rather than after, does not repair this: we construct a small explicit instance on which sign‑before (MuonUSign) and sign‑on‑both‑sides (MuonSign) ascend as well, so no placement of the sign around the oracle descends in general. Error feedback, the standard remedy for a biased compressor, does not rescue SignMuon: when applied to Muon's output, error feedback can fail for every smoothness constant, step size, and momentum. Applied to the gradient, error feedback does work, and EF21‑MuonUSign and EF21‑MuonSign attain the standard \mathcalO(T^‑1/2) rate for the squared gradient norm on smooth nonconvex problems, the latter at one bit in each direction. Experiments then reverse the ordering: across centralized CIFAR‑10, federated CIFAR‑10, and the nanoGPT speedrun, the strongest compressed method is consistently sign‑after‑the‑LMO, precisely the placement we prove divergent, with the provably convergent variants trailing it. Compressing after the LMO, a heuristic, matters more at these scales than the guarantee does.
Authors:Wenxin Tang, Jingyu Xiao, Zhenyu Liu, Zipeng Xie, Junliang Liu, Wang Luo, Yuan Jiang, Yintong Huo, Michael Lyu
Abstract:
Rendering source code as images offers a promising way to reduce the input costs of Multimodal Large Language Models (MLLMs). Adjusting image resolution can trade visual token cost against content fidelity. However, resolution scaling alone overlooks two sources of inefficiency: blank regions created by line breaks and indentation, and code regions irrelevant to the current instruction. Moreover, the best compression setting varies across inputs, tasks, and models, limiting fixed‑ratio strategies. We propose CodeShrink, an adaptive visual compression framework with three components. Blank‑Free Rendering replaces whitespace‑dependent layouts with compact layouts and explicit structural markers, removing layout‑induced tokens. Adaptive Compression Configuration uses a lightweight agent trained with reinforcement learning to predict a per‑input setting that balances token efficiency and readability. Dominant Token Selection jointly analyzes the instruction and code image to prune task‑irrelevant visual tokens during inference. We evaluate CodeShrink on code question answering, clone detection, and code completion. CodeShrink reduces visual token use by up to 71.2% while matching or exceeding uncompressed text‑only inputs, and consistently outperforms text‑based and visual compression baselines across all three tasks. These results show that combining layout compaction, adaptive configuration, and instruction‑aware pruning can make multimodal code understanding more efficient. Our code is available at https://github.com/vinsontang1/CodeShrink.
Authors:Zhisheng Han, Shiyao Wu, Jiayan Qiu, Yakun Ju, Lu Liu, Le Zhang, Pengfei Feng, Huiyu Zhou, Zheheng Jiang
Abstract:
Single‑image 3D hand avatar reconstruction is fundamentally ill‑posed and particularly challenging due to limited visual evidence under severe self‑occlusion and the complex pose‑dependent deformation of highly articulated hands. Existing methods predominantly rely on implicit NeRF‑style representations, whose volumetric fitting is computationally expensive and often struggles to preserve fine‑grained hand details. In this work, we present OASIS, a tailored 3D Gaussian Splatting framework for single‑image hand avatar reconstruction. To faithfully encode sparse image‑specific appearance cues in single‑view reconstruction, we construct geometry‑aligned visual evidence tokens by explicitly aligning input image observations with 3D hand geometry and context‑adaptively tokenizing the resulting visual evidence. Since severe self‑occlusion makes the reliability of image evidence inherently visibility‑dependent, we introduce a visibility‑conditioned point‑image attention to reliably transfer visual evidence to geometric tokens, yielding occlusion‑aware Gaussian features for faithful and robust reconstruction. To further capture non‑rigid deformation of articulated hands, we introduce a Feature‑on‑Mesh representation to enable Gaussian deformation to be guided by local surface stretching. Under this framework, we adopt a one‑shot adaptation scheme that learns a shared hand prior from multi‑identity training data and then fits it to a target image for target‑specific reconstruction. Extensive experiments show that OASIS outperforms existing baselines in both visual fidelity and efficiency across challenging poses and in‑the‑wild scenarios, and further demonstrates strong versatility in downstream applications such as text‑to‑avatar generation and texture editing.
Authors:Binnan Liu, Yechi Ma, Tian Xie, Wei Hua
Abstract:
The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input‑output examples and apply it to a new grid. Looped visual reasoners refine predictions over multiple iterations, but conventional training constrains only the final output, leaving intermediate refinements unconstrained. We propose that these refinements should instead follow the transformation step by step. We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains. We obtain these chains by rewriting and verifying programmatic task implementations, decomposing each solution into intermediate grid states. Each iteration is grounded by a task reference derived from the few‑shot demonstrations and an object workspace representing the current grid state. Because these chains may differ in length from the loop, soft trace alignment enforces only their ordering, letting the model allocate iterations freely. TraceViT achieves 67.8% pass@2 on ARC‑AGI‑1 and 24.3% on ARC‑AGI‑2. Controlled ablations on ARC‑AGI‑1 show that trace supervision becomes beneficial only when paired with grounding. Code and data will be available at https://github.com/LiuBinnan/TraceViT.
Authors:Boxiao Wang, Runxiang Wang, Kai Li, Chongming Li, Zhiwei Chen, Yifan Zhang, Jian Cheng
Abstract:
Symbolic Regression (SR) aims to discover analytical equations from observational data and plays a central role in scientific modeling. While recent Large Language Model (LLM) based approaches show promise, they face two limitations. First, they lack data analysis mechanisms for uncovering variable dependencies, which reduces the efficiency of equation discovery. Second, most methods rely on single‑objective evaluation focused solely on fitting error. This neglect of structural complexity and generalization often causes models to converge prematurely to local optima, limiting their ability to explore the broader equation space. We propose Multi‑Objective Tool‑augmented Symbolic Regression (MOT‑SR), a unified framework that integrates external analytical tools to extract structural priors and guide equation generation, while jointly optimizing for accuracy, complexity, and generalization via a multi‑objective evaluation module that maintains a dynamic Pareto front. MOT‑SR employs two collaborative LLM modules: a Meta Strategy Generator, which selects tools and synthesizes structural optimization strategies based on Pareto‑optimal equations, and an Equation Generator, which produces new candidate equations accordingly. The system operates in a closed‑loop manner, continuously refining both strategies and equation structures. Across 40 standard tasks, MOT‑SR outperforms existing SR methods in accuracy, generalization, and efficiency. We further validate MOT‑SR on extreme mass‑ratio inspiral (EMRI) orbital modeling, an important problem in space‑based gravitational‑wave astronomy where small local errors can accumulate substantially over long‑term evolution. The discovered interpretable correction achieves the lowest trajectory‑level integration error on held‑out configurations. These results demonstrate the potential of MOT‑SR to enable reliable modeling of long‑horizon scientific dynamics.
Authors:Shazzad Hossain, Proma Chowdhury, Mridha Md. Nafis Fuad
Abstract:
Evaluating evolving Natural Language Processing (NLP) models is important for ensuring reliable behavior across updates, but standard benchmark metrics do not fully capture how model behavior changes across versions. Existing work has focused mainly on testing models in isolation rather than comparing successive versions in continuous integration workflows. We present Alteron, a tool for detecting behavioral regressions across NLP model versions with metamorphic testing. Alteron constructs a test corpus from labeled source examples and compares model versions on metamorphically transformed inputs. In an evaluation spanning 10 metamorphic relations (MRs), 4 model versions, and 3 model‑update transitions, Alteron identified 16 behavioral regressions, 11 of which were release‑blocking. The results show that common model updates can preserve overall task performance while still introducing undesirable behavior changes, and that behavioral checks across model versions can reveal failures that aggregate benchmark metrics alone do not capture. The tool is open‑source and available at https://github.com/shazzad5709/alteron. A screencast demonstration is available at https://youtu.be/szwiWW5O4do.
Authors:Chong Gao, Jie Ma, Zhan Peng, Chongxiao Wang, Haoxue Wu, Jun Liang, Guanbin Li, Jing Li
Abstract:
Multimodal video generation aims to generate and edit videos conditioned on arbitrary combinations of text, images, and videos within a single model, allowing diverse tasks to share complementary data and generative priors. Unifying these tasks requires multimodal understanding of diverse conditions, which is typically provided by a pretrained vision‑language model (VLM). A key challenge is how to connect the VLM's hierarchical multimodal representations with a pretrained video diffusion transformer (DiT). Existing methods either inject features from only the final or a few manually selected VLM layers, or jointly train architecture‑matched understanding and generation streams, making it difficult to reuse heterogeneous pretrained backbones. We introduce MoRoute, a unified multimodal video generation framework that formulates a frozen VLM and a pretrained video DiT with different architectures as heterogeneous experts connected through dynamic layer routing. For each input, a lightweight block‑wise router enables every DiT block to select the VLM layer most relevant to its generation stage, thereby learning an adaptive correspondence between multimodal understanding and video synthesis. MoRoute further incorporates reference images and source videos directly into the DiT token sequence through unified in‑context conditioning, preserving fine‑grained visual details across diverse generation and editing tasks. Experiments on IntelligentVBench, OpenVE‑Bench, and RefVIE‑Bench show that MoRoute consistently surpasses the best competing method on each benchmark, improving the average score by 0.15, 0.18, and 0.34 on a 1‑5 scale, respectively.
Authors:Zhanpeng Zheng, Xiran Chen, Haiteng Jiang, Renjie Tian, Qinyu Cai, Jiexi Liu, Xiaofeng Chen, Weikai Li, Yansu Wang
Abstract:
Cross‑site identification of major depressive disorder (MDD) from resting‑state functional magnetic resonance imaging (rs‑fMRI) is hindered by inter‑site distribution shifts and heterogeneous functional connectivity (FC) views. These views capture complementary neural relationships but exhibit distinct site biases and graph topologies, complicating alignment without sacrificing disease‑relevant information or cross‑view consistency. Existing studies largely treat multi‑view connectome learning and cross‑site adaptation separately. To the best of our knowledge, few studies have jointly modeled multiple FC views under multi‑source unsupervised domain adaptation for cross‑site rs‑fMRI‑based MDD classification. We construct Pearson correlation, sparse representation, and Granger causality graphs, each encoded by a view‑specific graph attention network. Dual‑stream adaptive fusion explicitly integrates pairwise cross‑view interactions, followed by lightweight hyperbolic residual encoding for curvature‑aware representation refinement. Class‑wise Cauchy‑‑Schwarz alignment reduces inter‑source and source‑target discrepancies, complemented by adversarial learning, information maximization, and confidence‑aware pseudo‑labeling. Across seven unlabeled target domains, our framework achieves 73.60% mean accuracy and 71.90% AUC, demonstrating effective generalization under heterogeneous acquisition conditions. These results highlight the effectiveness of unified heterogeneous‑view modeling, curvature‑aware refinement, and multi‑source domain adaptation for cross‑site MDD identification.The source code is at https://github.com/OPUS‑Lightphenexx/MM‑HyperGDA
Authors:Zihao Liu, Xing Liu, Yizhai Zhang, Panfeng Huang
Abstract:
Driving style refers to the behavioral preferences that drivers maintain during driving, shaped by their diverse experiences, habits, and needs, and is typically reflected in varying levels of aggressiveness. If humans choose to use autonomous driving systems, they would expect the driving style of the systems to closely resemble their own habit. However, this is challenging for current industrial autonomous driving systems. To address this, we developed a style controllable action generation method, STAGE, for driving tasks. Its training process is based on imitation learning, incorporating both style value and latent value action modality encoding. Preference learning is then used to identify the user's driving style as a continuous, monotonic style value. And to reduce the cost of human involvement in the preference training process, we also developed a set of rules to compare driving style in data pairs. Then, during inference, the user inputs the style value to control the generated action patterns, dynamically meeting the user's expectations. Using the STAGE method, we verified that the style‑controlled action generation results in several typical road scenarios significantly align with human expectations. Furthermore, through comparisons between the STAGE method and various other approaches, we reveal the unique functionalities of STAGE, including its style controllability, style continuity, driving style alignment capability and driving safety. The code for this work is available at: https://github.com/CarlDegio/STAGE
Authors:Dylan Miller, Martin Jagersand
Abstract:
By relying on independent couplings from uninformative Gaussian priors, standard diffusion and flow matching models are forced to learn complex, high‑cost vector fields to reach the physical action space. Generative models excel at capturing multimodal behaviors for robotic Learning from Demonstration (LfD), but often suffer from high inference cost. This paper introduces Temporal Policy, a generative framework based on stochastic interpolants that formulates action generation as a temporally coupled transport problem. By initializing the generative flow at the robot's recent history, we explicitly couple past states to future action sequences. This data‑dependent coupling reduces transport cost and produces straight vector fields. We validate Temporal Policy across visuomotor simulation benchmarks and on a physical Barrett WAM 2x 7DoF teleoperation platform. Our approach reduces transport costs by nearly an order of magnitude compared to noise‑initialized baselines, achieving a 19.1 ms inference latency on a single NVIDIA RTX 4080. Crucially, these geometric and computational efficiencies are achieved while matching the success rates of state‑of‑the‑art baselines. This simplified transport geometry bypasses the computational bottleneck of independent Gaussian priors, helping enable high‑frequency, closed‑loop control. The code is publicly available at https://github.com/dmiller12/TemporalPolicy.
Authors:Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai, Haoyu Wu, Minghui Wu, Chenxu Zhao, Ante Wang, Guannan He, Changwei Wang
Abstract:
Self‑play agents can generate training problems without questions from target benchmarks, but their curricula lack persistent state: failures affect gradients yet do not explicitly shape future practice. External skill memories preserve procedural experience but are typically learned from fixed task distributions. We introduce SESA (Self‑Evolving Skill‑Augmented Agent), which makes procedural memory an evolving state of tool‑augmented search self‑play. A challenger poses problems, while a separately parameterized solver alone retrieves skills. Informative failures are distilled into reusable skills and written back to memory. The updated memory changes solver behavior and success, which changes the challenger's reward and the distribution of future problems; the resulting frontier produces new failures that rewrite memory. This bidirectional loop makes task generation and skill memory co‑evolve. Because retrieved skills shape on‑policy training trajectories, their benefits can enter the model parameters as well as remain in the external bank, enabling memory‑free deployment and optional inference‑time retrieval. Across seven open‑domain and multi‑hop question‑answering benchmarks, SESA improves average accuracy over SSP by 1.2‑‑3.2 points across multiple backbones and surpasses the skill‑augmented SkillRL baseline by 0.9 points under a unified evaluation protocol. On Qwen3 models, SESA‑Off retains 1.8‑‑2.2 points of improvement over SSP, while the final skill bank adds a further 0.5‑‑1.0 points. These results show that evolving skill memory is not merely an inference‑time plug‑in: it changes policy learning and the future training distribution while retaining value as optional external memory. Our code is available at https://github.com/Zenghuang‑Fu/SESA‑Self‑Evolving‑Search‑Agents.
Authors:Yilin Xiao, Zhehan Zhu, Yujing Zhang, Jin Chen, Zijin Hong, Luyao Zhuang, Qinggang Zhang, Shengyuan Chen, Xiaocao Ouyang, Lingfei Ren, Xiao Huang
Abstract:
LLM agents need memory to act consistently over long interactions, yet many systems use additional LLM calls to operate that memory. Generating intermediate records and mediating their retrieval adds recurring token and time costs, while omitted or merged details can obscure the original evidence. We ask whether structured memory access requires generation at all. Zero‑Mem introduces \emphzero‑token memory operations: no step outside final question answering invokes an LLM or consumes LLM input or output tokens; encoder computation is accounted for separately. Zero‑Mem preserves original interaction traces as its source of record. It organizes the traces in two complementary ways. An entity‑‑context graph exposes connections across interactions, while a temporal hierarchy preserves conversational locality and session state. For each query, Zero‑Mem weighs the two views, retrieves from both, and follows their structure to recover supporting relations or surrounding context. Deterministic calibration first discards conflicting evidence and then keeps the reader's answer grounded in the retrieved traces. Only the final‑QA reader invokes an LLM. Across long‑memory and long‑context question‑answering benchmarks, Zero‑Mem achieves competitive performance while eliminating LLM calls and LLM‑token consumption from memory operations. With the same final‑QA reader and context budget, it reduces memory‑operation time cost by 57.6% relative to the fastest compared baseline. Ablations support the contribution of the two views and their query‑dependent coordination. Overall, the results show that structured agent memory need not generate an intermediate representation of the past. After peer review, the code and implementation details will be available at \textcolorbluehttps://github.com/TheMoon0815/Zero‑mem.
Authors:Xueting Bai, Huan Ni
Abstract:
Existing cross‑domain few‑shot segmentation approaches suffer from high training costs due to source‑domain episodic training and pixel‑wise dense prediction, while often producing fragmented and noisy predictions. To overcome these issues, we propose a training‑free entity‑level few‑shot segmentation framework for remote sensing images with advection refinement. Specifically, we first leverage SAM3's generic geometric priors to generate category‑agnostic entity primitives. By reformulating few‑shot inference from pixel‑level prediction to entity‑level reasoning, foreground and background prototypes are constructed and combined with dense textual semantic responses from SAM3 to build a multi‑modal semantic potential field. Furthermore, an advection equation‑based semantic refinement mechanism is introduced to propagate category‑aware information across both feature and similarity spaces, enhancing semantic continuity and suppressing local texture noise. Extensive experiments on multiple remote sensing datasets demonstrate that the proposed framework effectively mitigates domain shift and local noise, substantially improving SAM3's adaptation capability for remote sensing few‑shot segmentation without additional training. Our code will be publicly available at https://github.com/yu‑ni1989/ELFSS‑AR.
Authors:Minghui Pan, Jiayuxuan Yang, Yuanyuan Yuan, Yu Jiang, Zhenpeng Chen
Abstract:
AI agents extend large language models (LLMs) with external tools, enabling them to perform complex tasks and translate model outputs into consequential real‑world actions. Yet LLMs often become substantially less safe when deployed as agents, and the source of this degradation remains poorly understood. In this paper, we identify schema‑formatted tool specifications as a primary source of agent safety degradation and show, through white‑box representation analysis, that they weaken the model's internal refusal signals and contribute to unsafe tool execution. Building on this finding, we propose SafeKeep, an inference‑time safeguard that decouples safety judgment from tool execution: it assesses requests using flattened textual tool specifications while retaining the original schema‑formatted specifications for execution. Across two representative benchmarks and four LLMs, including both white‑box and black‑box models, SafeKeep increases the average refusal rate for harmful requests from 23.8% to 70.6% and reduces the average attack success rate under observation‑level prompt injection from 25.6% to 2.5%. It also outperforms existing safeguards and preserves task‑handling capability. We release the code and data at https://github.com/snowcatsmoking/SafeKeep .
Authors:Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao, Shuwen Kang, Chi Lu, Wenjin Wu, Peng Jiang
Abstract:
Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training‑strategy changes. While LLM‑based agents can automate this trial‑and‑error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit‑Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long‑horizon exploration, RecHarness uses a jump‑basin mechanism to activate a structural‑jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM‑reasoning search. During a 7‑day online A/B test on a large‑scale short‑video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.
Authors:Roman Parpalak, Denis Utkin
Abstract:
We describe algorithms for the exhaustive enumeration and classification of simple arrangements of n pseudolines (n odd) maximizing the number of triangular faces. The depth‑first search enumerates reduced words for the longest permutation w_0 by branching only on the even‑indexed generators, using pruning constraints imposed by the geometry of optimal arrangements. The approach handles both perfect arrangements with a regular triangular pattern and unavoidable deviations from it for n \equiv 1 \pmod 6. The output is classified into a hierarchy of equivalence classes: by commutation, by Euclidean transformations, and by projective transformations. For each projective class we recover its full symmetry group G \subseteq S_n+1 together with the orbit‑stabilizer profile of its Euclidean subclasses. Completeness of the search and classification is proved: every wiring diagram is reached. We report full enumerations; e.g. for n=27, 85,562,064 wiring diagrams partitioned into 56,646 projective classes. For larger n (up to n=93), where exhaustive enumeration is out of reach, we report partial (first‑hit) results.
Authors:Xinyan Guan, Jiali Zeng, Chunlei Xin, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Fandong Meng
Abstract:
Large language models generate computationally expensive yet semantically void reasoning on beyond‑capability tasks, creating risks where plausible‑sounding but incorrect derivations mislead users. We characterize this futile reasoning phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce CaRL (Capability‑aligned Reinforcement Learning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability‑aligned behavior without sacrificing utility. \footnotehttps://github.com/icip‑cas/Knowing‑When‑to‑Quit
Authors:Weixiang Zhou, Xingguo Xu, Yuhao Wang, Cong Wang, Yang Yang, Zhixun Su, Jinshan Pan
Abstract:
Multi‑modal object Re‑Identification (ReID) aims to retrieve target instances by leveraging complementary information across modalities. However, existing methods suffer from two challenges. First, they often fail to exploit well‑aligned and reliable semantic priors, making them vulnerable to background clutter and cross‑modal misalignment. On the other hand, they typically rely on holistic feature modeling, overlooking the synergy between global and local representations. To overcome these limitations, we propose a robust multi‑modal ReID framework with dual semantic guidance and global‑local mutual modulation, which mainly consists of three key components, namely the Text‑Semantic Injector (TSI), the Masked Global‑Local Modulator (MGLM), and the Hierarchical MoE Fusion (HMF). The TSI enhances semantic awareness by integrating clean and coherent textual features into visual tokens. The MGLM enables part‑aware cross‑modal interaction through joint guidance from soft masks and global context, improving fine‑grained feature alignment. Finally, the HMF adaptively aggregates multi‑spectral features under local semantic supervision, yielding discriminative and robust representations. Extensive experiments on three multi‑modal ReID benchmarks demonstrate the effectiveness of the proposed method. The code will be made publicly available at https://github.com/zw‑absin/DSGM upon acceptance.
Authors:Pan Liu, Jing Li, Meng Zhao, Wanli Xue, Qinghua Hu, Shengyong Chen
Abstract:
With growing privacy and portability concerns, source‑free domain adaptation requires only a source pre‑trained model and an unlabeled target domain, allowing for effective adaptation to the target data. Most existing self‑training methods focus on selecting and exploiting samples with reliable predictions, often neglecting others. Inspired by the finding that deep models learn clean samples faster than noisy ones, we propose a domain‑division based progressive learning method named DPL. Specifically, our approach consists of two alternating stages, each beginning with the division of the target domain into easy‑to‑adapt and hard‑to‑adapt subdomains based on adaptation difficulty, followed by neighborhood‑based pseudo label assignment. In stage one, we enhance classification accuracy through uncertainty‑aware self‑training and alignment of corresponding classes between subdomains. Stage two then applies tailored learning strategies to each subdomain, starting with consistency learning on the easy‑to‑adapt samples and progressing to utilizing local structural information for the more challenging ones, thereby mining the intrinsic properties of the target data. Extensive experiments on several widely used benchmarks validate the effectiveness of our approach, demonstrating superior performance compared to state‑of‑the‑art methods. Our code is available at https://github.com/iamjingli/DPL.
Authors:Karim El Khoury, Benoît Gérin, Benoît Macq, Christophe De Vleeschouwer
Abstract:
Remote sensing scene classification is increasingly relying on foundation models pre‑trained on large‑scale Earth‑observation data. Moreover, transductive inference, which exploits the collective statistical structure of the entire unlabeled query set, appears to naturally match remote sensing pipelines where large images are routinely split into patches and inferred as a batch. In this work, we introduce LC‑TIM (Locally Consistent Transductive Information Maximization), which extends the state‑of‑the‑art Transductive Information Maximization for Few‑Shot CLIP (TIM++) objective with a local consistency regularizer that enforces prediction agreement between each query sample and its κ nearest feature‑space neighbors. The regularizer enters as a single multiplicative factor in the closed‑form q‑update, adding negligible computational overhead. We further propose a multi‑source extension that fuses the affinity graph from multiple remote sensing foundation model, further boosting classification accuracy. To assess these methods, we establish the first comprehensive, open‑source benchmark for transductive few‑shot RS scene classification, evaluating LP++, TransCLIP, TIM++, and LC‑TIM across ten diverse datasets, two remote sensing vision‑language models, and across various few‑shot settings. Our experiments show that transductive methods consistently outperform zero‑shot baselines, and that LC‑TIM achieves state‑of‑the‑art accuracy, with the largest gains in the low‑shot regime where neighborhood cues are most informative. Code is publicly available at: https://github.com/elkhouryk/LC‑TIM
Authors:Blaise Delattre, Cong Wang, Yang Cao
Abstract:
Tool‑using LLM agents act on typed tool returns, records pairing provenance and categorical fields with numerical values. Runtime permission gates generally authorize the observed return and action, leaving the decision unprotected against small errors in how the return was bound to its source. We ask whether a candidate action stays authorized over a declared neighborhood of plausible correctly bound returns: one admissible binding fault plus bounded numerical drift. We prove that certifying the categorical and numerical channels separately does not compose: perturbations that are safe on each channel alone can jointly turn the same action unsafe. CAGE certifies this joint neighborhood directly, enumerating the discrete branches exactly and certifying the continuous perturbation within each branch. Across synthetic, policy‑as‑code, regulatory, and real‑transaction settings, CAGE removes the in‑budget false allows that accurate pointwise gates admit, while keeping a useful fraction of decisions autonomous. When the policy is executable, CAGE‑Exact certifies the policy itself; otherwise CAGE‑Lip and CAGE‑RS certify a learned gate under an explicit, measured fidelity assumption.
Authors:Minju Seol, Minjee Seo, Seonaeng Cho, Kyungho Yoon
Abstract:
Transcranial focused ultrasound (tFUS) is a non‑invasive technique that delivers focused acoustic energy through the skull for neuromodulation and therapeutic applications. However, the heterogeneous structure of the skull induces complex, patient‑specific phase and amplitude aberrations that distort the acoustic focus and deviate it from the intended target, compromising therapeutic efficacy and safety. Conventional time‑reversal (TR) simulations can correct these aberrations but rely on computationally expensive full‑wave solvers, making them impractical for real‑time use and iterative treatment planning. We propose a few‑shot deep surrogate framework that predicts per‑element phase and amplitude corrections for a 96‑element 3D phased‑array transducer from patient CT images. A geometry‑aware encoder extracts skull‑path features shared across dedicated phase classification and amplitude regression branches, where phase periodicity is handled via circular expectation decoding. The framework is pretrained on diverse skull geometries and fine‑tuned with only ten target points, enabling rapid adaptation to unseen patients without full patient‑specific simulation. Evaluated via leave‑one‑out cross‑validation across 12 skulls, it achieves a mean phase CMAE of 0.155 rad and amplitude rMAE of 9.089%, a focal centroid error of 0.467 mm, Dice score of 94.422%, and peak pressure ratio of 92.332%, with an approximately 2,535 times speedup over TR simulation. The code is available at https://github.com/Minju‑Seol/fewshot‑tfus‑correction.
Authors:Wenda Yu, Tianshi Wang, Fengling Li, Xin Li, Jingjing Li, Lei Zhu
Abstract:
Vision‑language‑action (VLA) policies achieve strong performance in robotic manipulation but remain vulnerable to runtime disturbances that break the temporal alignment among visual observations, robot states, and executed actions. We introduce ActFovea, a plug‑and‑play safeguarding framework that detects and mitigates such failures without retraining or modifying the underlying VLA policy. ActFovea uses robot kinematics, proprioceptive states, and recent actions to construct action‑conditioned foveated regions that retain contact‑relevant areas and predicted motion corridors while suppressing task‑irrelevant visual content. It detects runtime risks by evaluating whether visual motion and observation freshness remain consistent with geometric, proprioceptive, and action transitions. For recoverable disturbances, ActFovea constructs disturbance‑specific candidate observations and accepts a recovery only after verifying the resulting action chunk. When stale or replayed observations make reliable recovery impossible, it invokes a bounded safe‑failure procedure. In closed‑loop evaluations of π_0 across multiple LIBERO suites, ActFovea increases success under localized visual overlays from 49.3% to 90.3%, closing 93.7% of the gap to clean performance. It further improves success under action drift and visual delay by 7.0 and 9.8 percentage points, respectively, while preserving clean‑task performance. Under frozen‑observation replay, ActFovea triggers timely safe failure in all trials, with no unprotected failures. These results demonstrate that spatiotemporal visual‑action consistency provides an effective basis for runtime safeguarding of VLA policies.
Authors:Zixuan Fu, Chong Wang, Lanqing Guo, Kailai Zhou, Jiahao Nie, Bihan Wen
Abstract:
Pixel‑space diffusion models aim to learn an end‑to‑end generator directly over raw pixels. This is challenging because a single model must capture both global structure and local texture in the same high‑dimensional space. While recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures, these advances typically require training a new model from scratch. We show there is a cheaper, complementary strategy: a frozen, pretrained pixel diffusion model can guide itself. Our key observation is that intermediate layers of a pretrained pixel diffusion transformer can be decoded into coarse predictions that capture the main low‑frequency structure, while the final layers progressively refine local, high‑frequency details. We therefore attach a lightweight prediction head to an intermediate layer, keep the backbone frozen, and use the discrepancy between the intermediate and final predictions as a self‑guidance direction during sampling. To train this head, we further find that real images are not necessary. Instead, model‑generated samples suffice and even outperform real images for training the head, especially in enhancing the high‑frequency components that pixel diffusion tends to underfit. Across multiple pixel diffusion models on ImageNet, our Synthetic Self‑Guidance (SSG) consistently improves generation while adapter training requires less than 1% of full‑model training compute: it reduces FID by over 50% across the evaluated JiT variants without classifier‑free guidance (CFG) and further improves strong baselines with CFG, e.g., JiT‑H/16 from 1.86 to 1.67 and PixelREPA‑H/16 from 1.81 to 1.59. Our code is available at https://github.com/zfu006/SSG.
Authors:Yongjie Zhou, Shuai Wang, Bevan Koopman, Guido Zuccon
Abstract:
Long‑term conversational agents require access to information from earlier interactions, such as a user's preferences, past requests, or previously mentioned facts. Repeatedly providing the full dialogue history can be expensive as conversations grow, so many memory approaches instead transform past interactions into compact entries that can be retrieved when needed. LightMem is a recent lightweight memory‑management approach that reports strong effectiveness while maintaining relatively low construction cost. However, it still relies on a separate constructed memory representation and is evaluated with only one retriever, leaving unclear how sensitive its results are to retriever choice and whether memory construction discards answer‑relevant information. In this study, we reproduce LightMem and compare it with Naive RAG, which retrieves directly from raw user turns. We recover LightMem's main configuration trend, but find that retriever choice is a major source of performance variation: changing only the retriever over a fixed LightMem store shifts answer accuracy from 58.1% to 75.5%. Constructed memories also do not consistently outperform raw‑turn retrieval. Naive RAG generally performs better at matched retrieval depths, whereas LightMem performs better mainly under tight answering‑token budgets. Oracle evaluation further shows that memory construction removes some answer‑relevant information. Overall, LightMem offers a context‑efficiency trade‑off rather than a general advantage over Naive RAG. Its value depends on the retriever and available token budget, motivating future work on retrieval, reranking, query formulation, and their interaction with raw and constructed memory representations.
Authors:Huiran Duan, Qian Zhou, Xianda Guo, Hua Zou, Guoying Zhao, Zhongyuan Wang, Yingli Tian
Abstract:
Gait recognition is shaped by its input representation. Silhouettes encode projected body shape, skeletons encode sparse joint coordinates, and 3D meshes encode dense surface geometry. In each case, identity‑bearing articulation is observed through geometric carriers that also vary with clothing, skeletal scale, or body shape. We investigate whether gait can instead be recognized from compact articulated controls. We introduce Momentum Human Rig (MHR) pose as a gait representation, describing each frame using 184 semantically organized body and hand parameters estimated from monocular video. MHRGait groups these heterogeneous controls by anatomy, models their intra‑frame coordination and temporal evolution, and produces compact body and hand descriptors. We further introduce MHRGait++, which combines MHR pose with silhouettes through modality‑balanced distance fusion, preventing descriptor count from determining modality importance. Experiments on four benchmarks show that MHRGait attains the best overall performance among compared model‑based methods on CCPG and SUSTech1K and transfers effectively across datasets, while its recognition network requires only 2.76M parameters and 0.69 GFLOPs for a 30‑frame input. MHRGait++ consistently improves silhouette recognizers with a favorable accuracy‑efficiency trade‑off. These results establish rig‑space articulation as an effective standalone gait representation and a complementary cue to projected body shape. Our code is available at https://github.com/duanhuiran/MHRGait.
Authors:Ning Hu, Haitao Duan, Shuqun Li, Chuyang Hu
Abstract:
Fractional scientific machine learning requires numerical operators that can be differentiated, batched, accelerated, and composed with neural networks. When the dominant linear fractional evolution is known through a Mittag‑Leffler propagator, repeatedly reconstructing that response with a history solver or relearning it from data is unnecessary. We present DFSC, a PyTorch environment organized around the Mittag‑Leffler Spectral Layer (MLSL). The layer separates known fractional propagation from data‑driven corrections, so neural modules learn only unresolved dynamics while fractional orders and residual‑network parameters are optimized jointly. Its adaptive algorithm increases special‑function truncation depth or Lanczos dimension until successive differentiable evaluations satisfy a requested tolerance. In the negative‑real alternating‑series regime, DFSC additionally returns a certified first‑omitted‑term bound; outside that regime it explicitly labels estimates as empirical. DFSC supports dense, sparse, matrix‑free, self‑adjoint, generalized, and controlled complex operator paths; trainable fractional orders; direct inverse problems; residual neural composition; and CPU/GPU execution. The certified series bound covers all 59 eligible reference cases, with median bound/error effectivity 1.246 for resolved errors. Reusing a prepared batched Lanczos basis gives identical fixed‑path values and reduces repeated‑query time by 4.61‑‑7.11 times on CPU and 13.07‑‑16.22 times on an RTX 5070, excluding one‑time preparation. A 27‑case inverse matrix finds full‑rank local curvature throughout, while remaining explicitly model‑conditional. External solver and mixed real‑data results support DFSC as an error‑aware optional primitive for matched fractional structure, rather than a general replacement for fractional solvers or neural models.
Authors:Yu Song, Hao Sun, Shiyu Teng, Ikuko Nishikawa, Yen-wei Chen
Abstract:
Existing methods for adapting 2D foundation models such as SAM to 3D volumes either process slices independently‑‑‑ignoring inter‑slice context‑‑‑or require substantial architectural changes and retraining. In this paper, we present SAM+D, a parameter‑efficient framework that lifts SAM‑family models by one spatial dimension‑‑‑enabling 3D volumetric segmentation from 2D SAM and, for the first time via parameter‑efficient fine‑tuning, end‑to‑end 4D (3D+T) spatiotemporal segmentation from video‑based SAM2‑‑‑while keeping the vast majority of pre‑trained parameters frozen. SAM+D introduces two lightweight, model‑agnostic modules into frozen transformer blocks: (1)~Depth‑Routed LoRA (DRLoRA) experts with learned routing for spatially adaptive low‑rank updates, and (2)~Depth Shift Modules (DSM) for cross‑slice feature exchange at zero additional parameter cost. Together, they provide volume‑level context while tuning only ~2.8% of parameters for SAM and ~3.7% for SAM2. We evaluate SAM+D in two distinct settings, each lifting the base model by one spatial dimension: 3D segmentation, where SAM(2D\,\to\,3D) is evaluated on four CT benchmarks (KiTS, Pancreas, LiTS, Colon), and 4D segmentation, where SAM2 (2D+T\,\to\,3D+T) is evaluated on a cell tracking challenge (CTC) dataset (Fluo‑N3DH‑SIM+). In both settings SAM+D achieves competitive or superior results under the single‑point prompt setting while using fewer trainable parameters than existing methods, demonstrating that SAM+D generalizes across SAM‑family architectures, target dimensionalities (3D, 4D), and domains spanning medical imaging and bio‑scene understanding. Code is publicly available at https://github.com/JerrySongCST/SAM‑Plus‑D.
Authors:Haodong Lei, Junming Liu, Yirong Chen, Pinlong Cai, Botian Shi, Ding Wang, Hongsong Wang
Abstract:
Large language model (LLM) agents increasingly operate over long interaction histories, where effective reasoning requires identifying and exploiting task‑relevant evidence distributed across past observations and actions. However, useful information encoded in previously computed representations is often underutilized during subsequent generation. We propose TransMem, a lightweight inference‑time parametric memory module that transforms sparse historical hidden states from a frozen LLM backbone into reusable memory representations. TransMem uses a lightweight gating network to dynamically apply the latent intervention to the current hidden states, without repeatedly encoding the preceding context. To learn transferable memory utilization rather than task‑specific knowledge, we introduce evidence‑conditioned self‑distillation. A memory‑augmented student processes the full context and matches the predictive distribution of an evidence‑only teacher that shares the same frozen backbone. Experiments on LoCoMo, HotpotQA, and MemoryAgentBench demonstrate consistent improvements across different model architectures and scales. TransMem yields gains of 11.58‑‑29.25 F_1 on LoCoMo and 10.20‑‑13.03 F_1 on HotpotQA, while improving the average MemoryAgentBench accuracy from 29.54% to 40.00%. These results establish sparse historical hidden states as an effective and efficient memory substrate for long‑context LLM agents. Our code is available at https://github.com/Haodong‑Lei‑Ray/TransMem.
Authors:Renxi Cheng, Jie Gui, Hongsong Wang
Abstract:
The rapid advancement of image generation models has made it increasingly difficult for people to distinguish AI‑generated images from real ones. To prevent the potential risks associated with the misuse of fake images, AI‑generated image detection has gained significant attention. Existing methods neglect the inherent differences between real and fake images, thus lacking robustness and generalization ability. In this work, we innovatively investigate AI‑generated image detection using bit‑planes, and introduce the bit‑reversed image. We propose a simple yet effective pipeline consisting of construction of bit‑reversed images, gradient‑based patch selection and a convolutional classifier. Besides, we provide a theoretical analysis from the mathematical perspective to demonstrate the validity of our approach. We also introduce two challenging datasets for AI‑generated image detection. Extensive experiments verify the effectiveness of our approach across different settings, including cross‑generator generalization, cross‑dataset generalization and zero‑shot performance. Without bells and whistles, our approach outperforms existing methods on over 40 benchmarks, and is nearly 100 times faster than counterparts. The code is at https://github.com/renxi‑seu/RAID.
Authors:Renxi Cheng, Chaolei Han, Jie Gui, Hongsong Wang
Abstract:
AI‑generated videos are becoming increasingly realistic and difficult to distinguish from authentic ones, which facilitates malicious misuse and poses growing threats to cybersecurity and social governance. Attributing AI‑generated videos to their specific generative sources is therefore of critical importance for forensic investigation and legal regulation. However, most existing visual attribution methods focus on images and particularly rely on the image generation model, thereby lacking the ability to generalize to large‑scale AI‑generated video data. To address these limitations, we introduce an training‑free AI‑generated video attribution paradigm. Specifically, we formulates AI‑generated video attribution as an instance retrieval task, and design a generative fingerprint‑based pipeline. This pipeline consists of an adapted orthogonal color transformation, multi‑scale quantized residual generation, and temporal‑semantic aggregation, progressively capturing and integrating artifacts introduced by generative models across video frames. Extensive experiments on the GenVidBench benchmark demonstrate that our method achieves strong performance in both AI‑generated video detection and attribution, outperforming existing state‑of‑the‑art methods with a Rank‑1 accuracy of 20.5% and a mean Average Precision of 16.6%. The code is at https://github.com/renxi‑seu/Video_Attribution.
Authors:Martin Lukk
Abstract:
Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on characteristics like race and gender. Recent LLM audits have produced inconsistent results, however, finding evidence of both positive and negative discrimination towards women and ethnic minorities, even for the same models. We show that this disagreement can arise from differences in audit format and introduce FairFund‑Bench, a benchmark that systematically varies key features of previous audit designs: the evaluation task (rating, ranking, or allocation), comparison context (single or multi‑stimulus), and whether the audit is transparent or disguised. The benchmark comprises 600 requests for financial assistance created from human‑authored templates (calibrated against 1.3M real GoFundMe campaigns) across three domains, four race and two gender categories, and five causal framings of need derived from welfare deservingness theory. Across 14 models, audit format changes the direction of bias: models advantage minorities when rating claimants individually but penalize some groups when ranking them side by side. Bias magnitude, though small overall, is several times greater in disguised audits than in transparent ones, where, faced with appeals differing only in claimants' names, models overwhelmingly split funds equally. Causal framing effects, by contrast, exceed demographic effects by roughly an order of magnitude and are consistent across models and audit formats, indicating that current LLMs robustly reproduce human deservingness evaluations. The benchmark scores models on four criteria (demographic bias, deservingness alignment, cross‑task consistency, and cross‑context consistency), is publicly available, and can be readily adapted to other substantive domains.
Authors:Xilin Tang, Yuqi Mai, William Kuszmaul, Alex Conway
Abstract:
Hash tables sit on the critical path of many systems, yet modern designs still force a trade‑off between fast operations and high memory overhead. We revisit this trade‑off and present Tiny Pointer Hash Tables (TPHT), a family of practical hash tables that make two ideas from theory work at system scale: compressing pointers down to a byte, and encoding keys compactly so less metadata is needed. We engineer these ideas into two complementary designs. Chained‑TPHT targets maximal space savings, and is to the best of our knowledge the first simple and practical succinct hash table design, achieving a footprint less than the total data size with constant‑time operations. Flattened‑TPHT targets latency, organizing data to keep the common case within a single cache miss while retaining strong space efficiency. Both variants support dynamic resizing without global pauses and integrate cleanly with 64‑bit keys and values. Across YCSB and microbenchmarks, TPHT advances the latency‑space Pareto frontier: Chained‑TPHT reaches 105.4% space efficiency, and Flattened‑TPHT achieves 83.4% space efficiency with up to 89.3% higher throughput than strong baselines. Together, these results show that techniques primarily known in theory can be turned into production‑ready hash tables that meaningfully reduce memory use while delivering state‑of‑the‑art performance.
Authors:Mohsen Seyedkazemi Ardebili
Abstract:
Cloud voice‑dictation services deliver strong accuracy but require streaming a user's speech to a remote provider, an unacceptable trade‑off in privacy‑sensitive professions and offline or air‑gapped settings; the leading on‑device alternatives are either platform‑locked or aimed at expert scripting rather than plug‑and‑play dictation. We present YazSes, an open‑source (Apache‑2.0) hold‑to‑talk voice dictation daemon that runs entirely on‑device, with a single codebase targeting Linux, macOS, and Windows through a protocol‑based platform abstraction. YazSes transcribes speech locally with faster‑whisper (CPU, int8) and injects the result into the focused application; a fast regex command grammar, backed by an optional small‑language‑model router, maps utterances to editor and terminal actions. Nothing leaves the machine: recording is push‑to‑talk rather than always‑listening, there is no telemetry, and an opt‑in personalization loop keeps its corpus encrypted on‑device and proposes configuration changes instead of shipping data out. We describe the system architecture ‑‑ a staged pipeline behind a protocol‑based platform abstraction with a JSON‑RPC control plane ‑‑ and its privacy and threat model. We evaluate the shipping Python implementation on a single commodity Linux laptop; the macOS and Windows backends are implemented and unit‑tested but not end‑to‑end evaluated here. On 200 LibriSpeech test‑clean utterances spanning 40 speakers, word error rate ranges from 4.82% (tiny.en) to 2.59% (small.en) at a real‑time factor of 0.520 for small.en, decoding faster than real time on CPU with no GPU. The command grammar reaches 100% action accuracy with a 0.0% false‑positive rate on plain dictation at 0.021 ms per call, and the non‑decode pipeline adds 0.289 ms of overhead. The system and the reproducible benchmark harness behind every number in this paper are public.
Authors:Fanzhe Wei, Li Liu, Ziyang Wang, Chenyu Wang
Abstract:
KV‑cache quantization is validated today by offline benchmark averages; a deployed system cannot tell whether compression is damaging the request it is serving right now. We give it a provably sound runtime meter ‑‑ a "DTrace for KV quantization": a per‑(layer, head, step) upper bound on the total variation between exact and compressed attention. The meter has two tiers: a deterministic band‑norm‑witness bound, sound for any cache‑preserving black‑box quantizer and for any query (adaptive‑safe, worst‑case Cauchy‑‑Schwarz plus RoPE band‑unitarity), and a tighter probabilistic certificate for a controlled subtractively‑dithered INT8 quantizer under an explicit request‑level failure budget (stated for non‑adaptive queries; core theorems machine‑checked in Lean 4). Three results. Observability: the meter enters SGLang through an env‑guarded patch, and any scheme registered as one tensor function is measured in live serving. Repair: meter‑driven gating ‑‑ risk‑ranked where the witness is saturated, certified where it is informative ‑‑ empirically restores the quality floor at benchmark scale, e.g. raw‑cast fp8 from 22.8 back to 79.7 on hard RULER tasks with the difference from uncompressed bounded at [+0.0,+0.8] by a paired test. Analysis: aggressive schemes survive on cross‑layer error cancellation, not per‑step fidelity ‑‑ in a 28‑layer sweep, no single layer's pollution alone loses anything (0/28) ‑‑ and the certified int8 cache serves 1.88× more KV tokens at the same memory in SGLang. All artifacts, guards, and the Lean development are released at https://github.com/metask‑ai/witcert‑kv‑certificates; every number regenerates from the shipped artifacts by one command.
Authors:Samurdhi Karunaratne, Anushka Idamekorala
Abstract:
We give a 55‑addition realization of rank‑23 multiplication of two arbitrary 3×3 matrices. Together with its 23 bilinear products, the circuit uses 78 scalar operations. This improves the previous state of the art of 56 additions, due to Sun. The construction starts from Perminov's public 58‑addition realization cr58_cn122 of a ternary tensor; the contribution is a shorter and, for this fixed orientation of that tensor, provably optimal linear circuit: 13 additions on the left input, 14 on the right input, and 28 at the output. The last circuit is obtained by transposing a 14‑addition factor circuit. Because the coefficient alphabet is \‑1,0,1\ and the order of every bilinear product is retained, the algorithm applies over every associative ring, commutative or not. We provide the full straight‑line program and tensor factors together with four exact computational checks, including independent Python and Node.js implementations of all 729 Brent identities over \mathbb Z.
Authors:Jiale Xu, Rendong Liang, Yuhao Long, Siyuan Shen, Zangyueyang Xian, Zeyi Xu, Yuanming Hu
Abstract:
Polygonal meshes are the standard surface representation of modern 3D pipelines, and generating high‑quality meshes with artist‑style topology is essential for film, gaming, and interactive 3D applications. Mainstream approaches serialize a mesh into a token sequence and decode it autoregressively, which is slow at inference and sensitive to error accumulation, making them impractical for interactive asset creation. We present Meshy T2, a fast native mesh generation framework built on flow matching. At its core is a vertex‑set mesh VAE that encodes a mesh into one continuous latent token per vertex and decodes vertices, edge connectivity, and face winding order in a single pass, preserving high‑precision geometry and artist‑authored topology without vertex quantization or welding. Generation proceeds as a coarse‑to‑fine cascade of two flow‑matching models: an image‑conditioned voxel flow first sketches the overall shape as a coarse occupancy scaffold, and a mesh flow then populates the scaffold with per‑vertex latent tokens, conditioned on the image, the scaffold, and a requested vertex budget. This design delivers three practical capabilities: interactive generation speed through parallel flow‑based synthesis; effective face‑count control through the requested vertex budget; and native support for multi‑part assets, whose components emerge directly from the generated connectivity. In our experiments, Meshy T2 achieves state‑of‑the‑art geometric fidelity and completes end‑to‑end image‑to‑mesh generation within a median of 6 seconds, over an order of magnitude faster than autoregressive baselines. Code and weights will be available at https://github.com/meshy‑dev/meshy‑t2.
Authors:Yuxuan Hu, Yuhao Wang, Tianbo Huang, Chao Zhang, Ziwei Liu, Lihua Zhang, Xiangyu Zhao
Abstract:
Cross‑domain sequential recommendation (CDSR) aims to model users' dynamic interest transitions and sequential patterns across multiple domains. Recently, generative recommendation (GR) has emerged. It first learns semantic identifiers (SIDs) from item semantics and formulates recommendation as autoregressive generation. However, existing methods face two critical issues: (1) they ignore collaborative correlations across domains during tokenization, and (2) they adopt inefficient decoding strategies, such as beam search, during generation, which hinders real‑time deployment. To address these limitations, we propose GenCDSR, an effective and efficient generative framework for CDSR. Specifically, we design a cross‑domain hybrid tokenization mechanism with a multi‑tower architecture to jointly capture cross‑domain commonalities and domain‑specific distinctions through hierarchical shared‑specific and fine‑grained codebooks. Furthermore, we develop a cross‑domain serial‑parallel decoding strategy that leverages the hierarchical SID structure to partially parallelize generation, significantly reducing inference latency while preserving generation consistency. Experiments on three public datasets show that GenCDSR achieves an average accuracy improvement of 1.5 percent and an average inference latency reduction of 85.1 percent compared with state‑of‑the‑art baselines. The implementation code and datasets are available online: https://github.com/Applied‑Machine‑Learning‑Lab/RecSys2026_GenCDSR.
Authors:Oliver Savolainen, Emanuele Bastianelli, Hosein Azarbonyad
Abstract:
Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential, which can be a barrier for non‑expert users. This work addresses the challenge by introducing a Task‑Aware Prompt Rewriter (TAPR), a model that reformulates user prompts into task‑optimized prompts with the explicit goal of improving downstream LLM performance. We train TAPR using reinforcement learning with Group Relative Policy Optimization (GRPO), where rewards are derived from LLM‑as‑judge evaluations of both the reformulated prompt and the corresponding task output. Experimental results on diverse tasks, such as question answering, summarization, and arithmetic reasoning, show that our method yields consistent gains over base models in prompt rewriting ability. Fine‑tuning Phi‑4‑mini‑instruct (as the base model for TAPR) produces prompts that contain clearer and more instructive language, leading to higher accuracy on established benchmarks such as Natural Questions and GSM8K. Our code is available at: https://github.com/OliverSavolainen/task‑specific‑prompt‑rewriter
Authors:Fan Wu, Cuiyun Gao, Yiming Huang, Yang Xiao, Yujia Chen, Qing Liao
Abstract:
Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshots to become a buildable codebase with shared components and working navigation. This project‑level setting exposes three limits of existing design‑to‑code benchmarks: they focus on single‑page generation rather than complete codebases, cannot evaluate cross‑page navigation, and do not measure project‑wide maintainability. We introduce MobileForge, the first benchmark for project‑level multi‑screen mobile app generation, comprising real mobile apps, human‑reviewed screens, structured page‑relationship annotations, and navigation test specifications. MobileForge supports five‑axis evaluation of build, navigation, visual fidelity, code maintainability, and efficiency. We also propose state‑isolated navigation testing to avoid cascading failures in navigation evaluation and an anchor‑referenced list‑wise visual evaluation protocol to improve visual‑judge reliability. Across end‑to‑end runs on six frontier multimodal LLMs, current models can build mobile‑app projects that compile and reach the correct pages, but interactive navigation remains unreliable and visual fidelity and maintainability still lag. The benchmark and supporting materials are available at https://github.com/anoa12159‑hue/mobileforge_eval.
Authors:Jonathan J. Heckman, Shani Meynet, Alessandro Mininno, Gary Shiu
Abstract:
Dualities play an important role in establishing both microscopic and emergent phenomena in a wide range of physical systems. In practice, though, it can often be computationally challenging to establish when two systems are dual, even when all of the "rules of the game" are well‑known. Said differently, when confronted with two systems, how can one efficiently establish that they are in fact dual? In this paper we use machine learning methods to address this question for Seiberg dualities of supersymmetric quiver gauge theories. Mathematically, this involves establishing mutations of quivers, which is in turn a variation on the theme of "learning to unknot". On the one hand, this leads us to a practical tool for establishing the computational complexity of different dualities. On the other hand, it also allows us to study how different network architectures learn how to trace Seiberg dualities. We find that for quivers with a modest number of quiver nodes (of order 10), different network architectures consisting of transformers and multi‑layer perceptrons tend to outperform deterministic algorithms. Supplementing the network by well‑established pathfinder algorithms (essentially "Google Maps for quivers") leads to an additional improvement in the efficiency and accuracy of the search strategy. We anticipate that this class of questions can serve as a useful benchmark for frontier AI models applied to theoretical physics.
Authors:Yao Xiao, Reuben Tan, Zhen Zhu, Yuqun Wu, Jianfeng Gao, Derek Hoiem
Abstract:
Long visual context poses a challenge for vision‑language models: performance degrades as the number of distractors grows, and processing all tokens at once is computationally infeasible under GPU memory constraints. We present ReToken, a single learnable embedding trained as an explicit retrieval target that selects a sparse set of query‑relevant visual tokens from a pre‑filled visual KV cache. Trained on only a small image‑QA dataset, ReToken yields consistent gains across image and video benchmarks: on Visual Haystacks it improves Qwen3VL‑8B by 13.4 points and InternVL3.5 by 12.4 points (>20% relative), and on LVBench it transfers zero‑shot to long video for an 8.0‑point gain with Qwen3VL‑8B. Thanks to its lightweight design, both training and long‑video inference fit on a single H100. Code is available at: https://github.com/avaxiao/ReToken
Authors:Yukang Cao, Haozhe Xie, Beichen Wen, Runmao Yao, Yinghao Liu, Yue Huang, Zhichao Liao, Yunxiang Wang, Haiheng Liu, Xingshun Tian, Dawei Su, Long Zhuo, Dacheng Tao, Xiaogang Wang, Liang Pan, Ziwei Liu
Abstract:
Embodied intelligence faces a fundamental data bottleneck. Models must capture how first‑person perception, whole‑body motion, dexterous manipulation, object state, sound, and touch evolve together as humans pursue goals over time. Existing datasets fragment this experience across viewpoints, modalities, or spatial scales, leaving the full perception‑action loop only partially observed. We introduce the Ambient Capture Engine (ACE), a human‑centric data engine that transforms real home environments into spatially calibrated, temporally synchronized recording studios. ACE operates at two complementary scales: a table‑scale configuration resolves hand‑object manipulation, while a room‑scale configuration captures whole‑body motion, locomotion, and interactions across a furnished home. ACE records egocentric and multi‑view exocentric video, full‑body and articulated hand motion, object geometry and 6‑DoF trajectories, audio, and tactile signals as a unified multisensory stream. Using ACE, we build ACE‑Data‑0, comprising 150 hours and 17M video frames across 200 task categories, performed by 50 participants in 2 environments, for a total of 75,000 interaction episodes. The dataset spans atomic manipulation, long‑horizon chains of household activities, and human‑scene interaction, while preserving natural behavioral variation through goal‑level rather than step‑by‑step instructions. We further introduce a hierarchical benchmark that progresses from signals to scene components and then to interactions. Evaluations of state‑of‑the‑art methods expose substantial gaps under contact, occlusion, egomotion, and long temporal horizons. ACE‑Data‑0 provides synchronized human demonstrations with aligned perceptual, kinematic, and contact supervision, offering a scalable foundation for imitation learning, world models, vision‑language‑action systems, and embodied AI.
Authors:Lizhi Yang, Junheng Li, Aaron D. Ames
Abstract:
We present PAC‑MAN, a perception‑aware CBF‑RL framework that couples control‑barrier safety with deployment‑realistic onboard sensing for whole‑body humanoid dodgeball. The deployed policy sees the ball only as segmentation‑masked depth from a head‑mounted camera, while training‑time CBF guidance represents clearance to every body link, and an adversarial motion prior regularizes the resulting evasive reflexes. We evaluate on a controlled any‑link contact benchmark with seeded throws in two regimes: single throws and a deployment loop in which the robot walks back to its station and recovers between throws. On this benchmark, the policy comes within a few points of a privileged state oracle: a fixed onboard camera alone is adequate for evasion. We find that usable barrier structure depends on perceptual observability: Joint‑CBF gives the best performance with accurate ball states, degrades under fixed‑camera observations when used only as training guidance, and recovers with a ball‑tracking gimbal or privileged runtime filter. We therefore deploy a lightweight Link‑CBF policy zero‑shot on the Unitree G1 in the real world, where it tolerates imperfect perception, succeeds on 95% of throws, and uses semantic segmentation to dodge different balls.
Authors:Qiushi Sun, Kanzhi Cheng, Yian Wang, Bowen Yang, Hang Yan, Liheng Chen, Fangzhi Xu, Zichen Ding, Nuo Chen, Jialin Cao, Xingdong Gong, Zehao Li, Kaiming Jin, Xinfeng Yuan, Zhoumianze Liu, Jingyang Gong, Zhangyue Yin, Jiahui Gao, Zhiyong Wu, Tianbao Xie, Jianbing Zhang, Ben Kao, Lingpeng Kong
Abstract:
Computer‑using agents (CUAs) are advancing rapidly across the digital world. A CUA trajectory records the agent's actions, states, and reasoning. Verifying whether it fulfilled the task instruction is central to CUA evaluation, data curation, and reinforcement learning. Neither human‑written verifiers nor human annotators can provide such verification at scale, so the field increasingly turns to vision‑language models (VLMs) as judges of CUA trajectories. But a fundamental question has long gone unexamined: are these VLM judges reliable enough? To study it systematically, we introduce OSReward, a realistic, high‑quality benchmark that evaluates VLM judges on CUA trajectories. The trajectories come from diverse agent backbones executing human‑verified instructions across platforms, and are then rigorously labeled with ground‑truth verdicts through multi‑stage human annotation. Building on it, we derive OSReward‑Hard, a challenge set concentrating genuinely hard cases, and OSReward‑Multi for fine‑grained efficiency and alignment scoring. The most comprehensive evaluation of VLM judges to date finds even state‑of‑the‑art models fall short of an ideal judge, sharing a systematic leniency bias that mislabels failed runs as successes. The few reliable enough to trust are too expensive to run at scale, while affordable open models trail far behind. To close this gap, we construct and release OS‑Shepherd‑100K, an open corpus of reasoning‑annotated trajectory judgments for the CUA community. On it, we train OS‑Shepherd (9B and 35B), open reward models that supply low‑cost, stable, and reliable reward signals, matching commercial judges at 30‑60x lower cost than the frontier. Extensive analyses further inform the design of reliable CUA reward at scale. Our code, benchmark, dataset, and model checkpoints are available at https://os‑copilot.github.io/OSReward‑Home/.
Authors:Kangning Zhang, Yixing Li, Shuai Shao, Qingyao Li, Zhengxi Lu, Zhiyuan Yao, Jianghao Lin, Wenxiang Jiao, Yuan Lu, Weiwen Liu, Weinan Zhang, Yong Yu
Abstract:
Multimodal on‑policy distillation (OPD) transfers fine‑grained visual knowledge by supervising student‑generated trajectories with a privileged‑view teacher. Yet its next‑token corrections are source‑mixed, combining visual signals with linguistic priors and teacher‑specific effects. The key challenge is to estimate which corrections are supported by visual evidence, not merely where or how strongly to distill. We introduce Visual Attribution Distillation (VAD), a counterfactual target‑reconstruction algorithm that estimates the visually attributable part of a teacher correction. At each student‑generated prefix, VAD evaluates the same fixed teacher with the relevant evidence present and removed. The corresponding change in centered log‑probabilities defines ut, a signed proxy for the visual evidence direction that estimates how revealing the evidence supports or refutes candidate tokens. VAD projects the original correction onto this proxy to obtain an intervention‑aligned component and a proxy‑unexplained residual, then reconstructs a student‑anchored target from the former. During training, this reconstructed target supplies the primary supervision signal, while the privileged teacher contributes a weak regularizer. Across six fine‑grained visual benchmarks at 4B and 9B scales, VAD outperforms direct privileged‑view distillation and visual‑advantage weighting. Token‑ level and controlled‑target analyses show that the proxy‑aligned component is enriched in task‑relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer. These results support counterfactual target reconstruction as an effective alternative to source‑mixed supervision.
Authors:Manyi Wang, Junjielong Xu, Pinjia He
Abstract:
SWE‑bench‑like benchmarks are widely used for evaluating LLM's issue resolution capability. They typically follow a common construction pipeline: each PR (Pull Request) is paired with its linked issue by extracting issue references from the PR description; the issue description is used as the problem statement, and the PR patch serves as the test oracle. However, due to the inherent complexity of developing and maintaining large repositories, such PR‑Issue pairings are often misaligned in practice. In this work, we systematically study SWE‑bench Verified instances, finding that 13.6% exhibit misalignment across five patterns in eleven fine‑grained scenarios. To enable reliable and scalable construction of those benchmarks in the future, we propose PAIChecker, a multi‑agent system for checking PR‑Issue misalignment in SWE‑bench‑like benchmarks. Specifically, PAIChecker adopts a three‑phase design that combines specific pattern identification, cross‑agent label synthesis, and code‑level validation, thereby enabling more accurate, generalizable, and progressively verified detection. Experiments on SWE‑Gym and SWE‑bench Multilingual show that PAIchecker achieves the best performance across all four LLM backbones, reaching up to 92.12% and 91.67% binary accuracy, respectively.
Authors:Xiao Luo, Mingyang Du, Xin Zhou, Tianrui Feng, Xiwu Chen, Xiaofan Li, Jiangning Zhang, Dingkang Liang
Abstract:
High‑fidelity 3D generation predominantly relies on scaling model capacity and data, which incurs prohibitive computational costs. This paradigm typically requires learning geometry from scratch and overlooks the rich semantic and structural priors already encapsulated in discriminative 3D foundation models. We contend that leveraging the profound understanding of the 3D world possessed by these discriminative models can significantly reduce generative cost. To this end, we propose ROAD, a framework that reduces the training cost of 3D generation by transferring these rich discriminative priors into diffusion transformers. To address the inherent semantic‑structural heterogeneity between generative and discriminative latents, we introduce a reciprocal‑objective alignment strategy. This method synergizes Holistic Semantic Condensing to enforce global semantic coherence and Structural Optimal Alignment, which is formulated as a bipartite matching problem to rigorously align microscopic geometric details between disparate latent spaces. The 3D foundation model is only used for training‑time supervision of alignment and is not used at inference, incurring no additional inference cost. Compared with the industrial baseline Step1X‑3D, the proposed ROAD achieves highly competitive generation performance with only 1.5% of the training data and significantly reduces training costs, effectively reducing the computational overhead of high‑fidelity 3D generation. Code is available at https://github.com/H‑EmbodVis/ROAD.
Authors:Junlin Yang, Che Jiang, Yu Fu, Tianwei Luo, Can Ren, Weizhi Wang, Kaikai Zhao, Hongyi Liu, Yuxin Zuo, Yuru Wang, Yuchen Fan, Kai Tian, Zhenzhao Yuan, Xiaojian Lin, Li Sheng, Rushi Qiang, Guoli Jia, Xingtai Lv, Ermo Hua, Dianqiao Lei, Youbang Sun, Ning Ding, Bowen Zhou, Kaiyan Zhang
Abstract:
Recursive self‑improvement (RSI) requires AI systems that improve the process of building AI (i.e., AI4AI); machine learning engineering (MLE) offers a concrete, executable testbed for studying this capability. We introduce OpenMLE, an open full‑stack system for RSI research in MLE, spanning verifiable task environments with execution feedback (OpenMLE‑Gym), operator learning (OpenMLE‑RL), and long‑horizon search (OpenMLE‑Evo). On this stack we post‑train Frontis‑MA1 (35B) as a meta‑evolution agent for MLE, aligning post‑training and inference around four atomic program‑evolution operators (Draft, Improve, Debug, Crossover): the same operators are trained via execution‑grounded SFT and RL on data deduplicated against all evaluation benchmarks, then composed into long‑horizon search, coupling learning and evolution in a single loop. On MLE‑Bench Lite under a 12‑hour per‑task budget on one RTX 4090 capped at 12 GB VRAM, Frontis‑MA1 (35B) improves Medal Average from 39.39% to 60.61% over its base model with OpenMLE‑Evo, and reaches 71.21% with OpenMLE‑Evo‑Max (benchmark‑independent experience priors and asynchronous search), exceeding GPT‑5.5 + Codex and approaching GPT‑5.6 Sol and the 2.8T Kimi K3. On held‑out NatureBench Lite, both components transfer: with the framework fixed, swapping in the trained model raises Match‑SOTA from 50% to 70%; with the model fixed, swapping in OpenMLE‑Evo raises it from 20% to 50%. We release the model weights and the full OpenMLE stack to enable reproducible research on executable AI4AI toward RSI. Code: https://github.com/FrontisAI/OpenRSI
Authors:Tianyu Yang, Yiming Zeng, Wenzhe Cai, Yuqiang Yang, Jiaqi Peng, Hui Cheng, Jiangmiao Pang, Tai Wang
Abstract:
Pretraining navigation diffusion policies rely on large‑scale expert demonstrations. These data are typically generated by a fully‑informed oracle planner suited to a single nominal robot. This limits the policy's generalization to diverse embodiments and challenging scenarios (e.g., escaping dead ends or detouring long obstacles) that demand diverse local reactive behaviors with only onboard local observations. Post‑training the policy with reinforcement learning (RL) offers a principled remedy. However, previous RL for diffusion approaches lead to only marginal improvements. This is because the intractable likelihood of diffusion policies renders policy gradients unstable in addition to inefficient policy exploration. To address these challenges, we propose a data‑efficient diffusion RL post‑training framework ‑ GQRM (Group Q‑score Reweighted Matching). Our framework introduces two complementary designs: (i) a self‑bootstrapped exploration strategy with behavior perturbation that preserves the pretrained policy prior, and (ii) a group Q‑score normalization mechanism that computes per‑trajectory values on each state for efficient reweighted score matching. By conducting distributed online RL training across heterogeneous embodiments, the resulting fine‑tuned policy, X‑NavDP, achieves state‑of‑the‑art cross‑embodiment visual navigation performance, improving the overall success rate from 61.20% to 84.28% in simulation and 10% to 65% in real‑world hard cases. The code and model are publicly available at https://yty‑sky.github.io/x‑navdp‑project‑page.
Authors:Tengfei Liu, Yang Shi, Yuran Wang, Xiaohan Zhang, Yuqing Wen, Yuqi Tang, Qixun Wang, Zhuoran Zhang, Xuanyu Zhu, Weihong Lin, Xinlei Yu, Yujie Wei, Xinwei Long, Fengxiang Wang, Xinlong Chen, Yue Ding, Jialu Chen, Haotian Wang, Yuanxing Zhang
Abstract:
Existing video captioning models generate natural descriptions of video content but cannot explicitly ground local visual elements to multiple reference images. We introduce multi‑reference image‑grounded video captioning, a new task requiring factual video descriptions with phrase‑level reference grounding, and propose RefCaptioner, a two‑stage post‑training framework for this task. RefCaptioner combines mixed‑data SFT with Hierarchical Coverage‑Discounted GRPO to jointly improve reference selection, phrase‑level binding, distractor rejection, and cross‑reference consistency while preserving general video‑captioning ability. To support training, we construct a corpus containing 20,000 videos and 171,354 reference images. We further introduce MRVBench, a benchmark for evaluating caption factuality and multi‑reference grounding on both real‑world and AI‑generated videos. Experiments show that RefCaptioner achieves the best overall performance among the open‑source models while remaining competitive on standard video captioning benchmarks. Human evaluation further confirms that its captions are preferred by annotators and enable more source‑faithful video reconstruction with both open‑source and proprietary video generators.
Authors:Zheng Wu, Chenhao Xue, Shijie Zheng, Yijie Lu, Cheng Yang, Zhuosheng Zhang
Abstract:
As large language models (LLMs) continue to advance in complex reasoning tasks, they have learned to heavily prioritize explicit conditions provided in the input. However, in everyday commonsense reasoning, this mechanism exposes a critical vulnerability which we term Salience Bias: models become easily hijacked by useless explicit distractors (e.g., numerical values), leading them to ignore the implicit physical or commonsense prerequisites of a task. A critical open question is whether this failure reflects a genuine gap in commonsense knowledge or merely its suppression under misleading task framing. To investigate this, we construct the SaliTrap Benchmark, a high‑quality dataset across four trap dimensions. Evaluating 12 state‑of‑the‑art LLMs, we find that all mainstream models suffer significantly from salience bias, with severity scaling with distractor density and detecting the trap often decoupled from actually avoiding it. Crucially, by re‑eliciting the same models with the task framing stripped away, we show that this is overwhelmingly a failure of knowledge suppression rather than knowledge absence: a context‑free knowledge probe alone recovers over 90% of sycophantic‑compliance failures, revealing that the requisite commonsense is intrinsically present but actively crowded out by salient distractors that lure the model into over‑compliant, unnecessary computation. Building on this diagnosis, we further show that lightweight, inference‑time prompting alone substantially closes the gap without any retraining. Our findings relocate the bottleneck of commonsense reasoning failures from model competence to elicitation, and we release SaliTrap as a testbed for this blind spot. The codes are available at https://github.com/Wuzheng02/SaliTrap.
Authors:Xiaobei Zhao, Xingqi Lyu, Xin Chen, Xiang Li
Abstract:
Vision‑and‑Language Navigation in Continuous Environments (VLN‑CE) requires an agent to follow a natural language instruction, predicting a sequence of low‑level actions to navigate a robot from a starting point to a target location. The A2A benchmark and the AgriVLN method pioneeringly extended VLN‑CE from indoor scenes to agricultural scenes, while we observed a challenging distinction: In indoor scenes, whether a zone is traversable tends to be clear to classify, such as wood floors are traversable but concrete walls are not. In agricultural scenes, however, this issue tends to be ambiguous, such as an unripe cornfield might be traversable for a robotic dog but might be non‑traversable for a human. To address this issue, we propose the TEA module, which estimates the traversability of the camera image, then alarm the decision‑maker for rethinking when the predicted action does not align with the traversability map. We integrate it into the AgriVLN backbone to build our TEA‑AgriVLN method. When evaluated on A2A, it improves Success Rate (SR) from 0.47 to 0.54 and Navigation Error (NE) from 2.91 m to 2.70 m, showing the state‑of‑the‑art performance in the agricultural VLN‑CE domain. We further implement the ablation studies and the case study, discussing the effectiveness and limitations of TEA on different ground categories and scene classes. Code: https://github.com/AlexTraveling/TEA‑AgriVLN.
Authors:Zheng Wu, Yibo Luo, Pu Zhang, Cheng Yang, Zhuosheng Zhang
Abstract:
Generative UI (GenUI) lets large language models synthesize a complete, renderable interface directly from a natural‑language instruction, but evaluating the quality of what they generate remains an open problem. Human evaluation is costly and rater‑variant, while LLM‑as‑a‑judge is scalable but reflects only a single implicit viewpoint, unable to capture how different populations of real users actually perceive the same interface. We propose the Evidence‑Grounded, Social‑Weighted Persona Panel (ESPP), a three‑stage GenUI evaluation method in which a panel of psychologically diverse, evidence‑grounded personas independently rates a screenshot, exchanges opinions under a trait‑derived, semantically‑gated bounded‑confidence mechanism, and is aggregated via Delphi‑inspired social weighting into a single judgment. ESPP tracks human judgment substantially more closely than a naive single‑pass judge, raising Pearson r from 0.716 to 0.922, and a prompt‑ensemble control recovers only about a third of this gap, isolating genuine persona and evidence grounding as the dominant source of improvement. Beyond this fidelity gain, retaining each panelist's individual rating further reveals that user subgroups agree on overall model rankings yet diverge sharply on specific rating dimensions, a structural disagreement a single homogeneous judge would systematically erase. The codes are available at https://github.com/Wuzheng02/ESPP.
Authors:Xinxing Ren, Qianbo Zang, Ziyan Wang, Caelum Forder, Suman Deb, Peter Carroll, Zekun Guo
Abstract:
Understanding large codebases is a long‑horizon task for Large Language Model (LLM) agents: answering a single question can require building and running the software, tracing execution across files, and synthesizing evidence over tens of minutes. On SWE‑Atlas QnA, a benchmark of long‑horizon questions over production repositories, a single Claude Code agent (Opus 4.6) resolves only 32.3% of tasks. Dividing the work among agents with clean contexts mitigates this limitation. However, the subtasks of code comprehension are interdependent. One agent's findings can rewrite another's task, so agents must coordinate during execution, not only at phase boundaries. Existing multi‑agent systems support such exchange only between phases, through staged handoffs or synchronized rounds. Communication and work remain mutually exclusive. A discovery made mid‑execution cannot be shared until the next boundary. We present AgentRadio, an asynchronous message‑passing layer that equips coding‑agent harnesses with three primitives: threads, messages, and waiting for mentions. The last runs as a background task, surfacing teammates' messages without interrupting foreground work, so each agent remains passively aware of its peers and folds new findings into its ongoing task. Under a five‑phase protocol of division of labor and negotiation, four agents organized by AgentRadio resolve 62.1% of tasks, 29.8 points above a single agent and above Claude Code with the newer Opus 4.8 (57.2%). Rubric‑level analysis shows the gain growing with task difficulty, consistent with mid‑course correction as the underlying mechanism. Our code is available at https://github.com/Coral‑Protocol/AgentRadio.
Authors:Zongheng Guo, Tao Chen, Tianli Li, Mingzhe Cui, Yang Jiao, Lei Xie, Yi Pan, Xiao Hu, Manuela Ferrario
Abstract:
Derived measurements increasingly enter large language model (LLM) pipelines as direct facts despite their instance‑dependent validity. We define derived‑feature over‑trust (DFOT) as the failure in which a downstream LLM assigns such a measurement the epistemic status of a direct fact or uses it outside its valid scope. Using physiological sensing as a case study, D1 tests acceptance of a PPG‑derived rhythm contradicted by offline ECG, whereas D2 tests rejection of an offline‑confirmed reliable PPG rhythm under misleading severe history. ECG supplies training supervision and offline reference construction but is never shown to the LLM. Five estimands quantify this chain: conflict over‑trust rate (COTR) and context‑induced error rate (CIR) characterize D1/D2; correct repair rate (CRR) measures frozen‑error repair; evidence‑specific repair margin (ESRM) contrasts matched and patient‑disjoint shuffled evidence; and utility harm rate (UHR) measures unnecessary verification among HIGH‑reliability cases used without verification at baseline. The framework does not depend on a particular reliability generator. We demonstrate it on 50,000 paired PPG‑ECG records using ECG‑to‑PPG privileged distillation as an illustrative baseline and PPG‑only inference. On a protocol‑locked 187‑patient test, the baseline improves four repair and specificity endpoints by 1.82‑6.69 percentage points, with all paired confidence intervals excluding zero; UHR increases by 0.67 percentage points (95% CI: ‑0.4 to +1.7). DFOT provides a common evaluation target for stronger mitigation methods. The code is available at https://github.com/Zongheng‑Guo/When‑Derived‑Measurements‑Mislead.
Authors:Haozhe Hu, Hao Wu, Peiran Yin, Chao Han, Yunpu Ma, Xiaoyu Shen
Abstract:
Pruning is a promising approach for improving the efficiency of LLMs. Existing static structured pruning methods are hardware‑friendly and can deliver practical throughput gains, but their input‑agnostic computation allocation often causes substantial accuracy degradation under aggressive sparsity. Recent dynamic sparsity methods improve quality retention by adapting computation to individual inputs, yet they remain largely limited to coarse‑grained structural decisions and their practical acceleration under real‑world inference scenarios remains challenging. To address these challenges, we present WIDE, the first end‑to‑end differentiable token‑level dynamic width pruning framework designed for both prefill and decode scenarios. WIDE enables fine‑grained computation allocation by allowing each token to dynamically select attention‑head groups and FFN‑channel groups, extending dynamic pruning beyond layer‑level decisions to neuron‑block‑level granularity. Through a two‑stage training pipeline, WIDE learns effective token‑wise sparse execution patterns and achieves substantially better quality retention than existing approaches. To make such fine‑grained dynamic pruning practical, we further propose a pruning‑‑kernel co‑design framework that decomposes dynamic sparsity acceleration into mask reordering, hardware‑agnostic block‑level skipping, and hardware‑dependent intra‑block skipping, enabling efficient execution across different granularities. At 50% sparsity, WIDE provides 55.1% performance boost when compared to the state‑of‑the‑art dynamic depth pruning under calibration‑only settings. Under prefill and decoding inference workloads, WIDE achieves close‑to‑theoretical kernel‑level speedups of up to 1.98x for prefill and 4.95x for decoding, as well as 1.68x and 1.55x end‑to‑end acceleration. Our code is available at https://github.com/EIT‑NLP/LLM‑Pruning/tree/main/WIDE.
Authors:Mingkang Dong, Muxin Pu, Jie Li, Bohan Guo, Songruo Chen, Bin Ren, Xu Zheng, Chen Zhao, Tianwen Qian, Mohamed Elhoseiny, Yuqian Fu
Abstract:
Streaming video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to token importance, temporal redundancy, or segment‑level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training‑free framework that treats latent objects as memory anchors for streaming video understanding. ObjectStream induces spatially coherent latent objects directly from frozen Video‑LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded memory budget, without requiring external object detectors or segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing Video Large Language Models (Video‑LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long‑video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5‑VL‑7B by 10.0 points on OVO‑Bench Real‑Time Visual Perception, while reducing peak GPU mem‑ory and TTFT by approximately 50%. On offline long‑video benchmarks, it surpasses the full‑token baseline while discarding 82.5% of visual tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming video memory.
Authors:Jens Lehmann, Andrei Aioanei, Sahar Vahdati
Abstract:
ARC‑AGI‑3 turns abstraction into an interactive problem of skill acquisition. A player must infer an unfamiliar game's rules, hidden state, and goal while maintaining action efficiency because every move counts. We formalize these environments as parameterized rendered deterministic Moore machines and introduce Tycho, a coding‑agent system that constructs and uses game‑specific models during interaction. Tycho separates actionable observations from intermediate animation, level‑completion, and game‑over frames. From this structured history, an agent can model, test, plan with, repair, or bypass a free‑form executable hypothesis. In one matched public‑set run per policy, we compare four orchestration policies on all 25 public games using Claude Opus 4.8 under matched inference budgets. Actor‑requested delegation to a model builder obtains the highest observed mean Relative Human Action Efficiency (RHAE), 88.49. With this selected policy, GPT‑5.6 Sol and Opus 5 both reach 100.00 RHAE and complete all 183 levels. Their game‑balanced first‑run human‑replay midranks are 98.5 and 100.0. Opus 5 uses 61% fewer scored actions than the aggregate official human baselines. Automatic repair after verification failures produces models that reproduce observed transitions much more accurately, yet reaches only 83.07 RHAE. Transition match indicates whether a simulator reproduces observed dynamics, not whether it has identified the objective or improves the next action. Strong play also requires deciding when to construct, repair, use, or bypass a model. We call this joint problem active abstraction: generating a testable model from costly interaction and deciding when acquiring or using it is worth its cost.
Authors:Jiwen Liu, Shujuan Li, Xiaohan Li, Zijie Meng, Xinyue Liu, Yulong Xu, Yan Zhou, Guoxin Zhang
Abstract:
Video re‑shooting aims to regenerate videos with controllable camera motion and viewpoint. Existing methods rely on explicit 3D priors, which are limited by reconstruction quality and often perform poorly when synthesizing previously unseen regions, or on paired videos with different camera trajectories, whose scarcity hinders generalization. We revisit video re‑shooting through text‑driven semantic viewpoint specification, enabling control over shot scale, viewing angle, and first‑/third‑person perspective. To this end, we propose TARS, a 3D‑free video re‑shooting paradigm. Timestep‑wise sensitivity analysis reveals that camera motion is primarily established during high‑noise stages, where coarse spatiotemporal structures are formed. Based on this insight, we introduce self‑supervised training to learn camera dynamics and fundamental visual representations without paired re‑shooting data or 3D reconstruction. Through data scaling and joint textual‑camera conditioning, TARS supports robust camera and viewpoint control, plausibly synthesizing regions beyond the source view under large camera motions while enabling reverse‑angle re‑shooting and perspective switching. Extensive experiments show that TARS provides more accurate and temporally consistent camera control than prior methods. Project Page: https://ymlinfeng.github.io/TARS.github.io/
Authors:Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand, Majid Ramezani
Abstract:
Pre‑trained language models have significantly improved sentence representation learning, yet their embedding remain sensitive to semantic preserving textual perturbations such as synonym substitution, masking and word dropout. This work proposes a lightweight Contrastive Denoising Autoencoder (CDAE) that refines pre‑trained BERT embedding by jointly optimizing contrastive and reconstruction objective to learn perturbation‑invariant representation. We evaluate the proposed framework using multiple perturbation strategies with varying strengths and compare it against the original BERT embeddings and SimCSE. Experimental results show that CDAE consistently preserves higher embedding similarity under perturbations, with the improvements becoming more pronounced as framework effectively enhances representation stability while preserving semantic information, highlighting perturbation‑invariant learning as a promising direction for improving sentence embeddings. The source code is publicly available at: https://github.com/ComputationIASBS/CDAE
Authors:Luigi Sigillo, Matteo Silvestri, Francesco Tabaro, Rajat Bhatnagar, Syed Irtaza Mubashar, Matt Jeffryes, Daljit Nijjer, Vittorio Perera, Ola Spjuth, Julio Saez-Rodriguez, Melissa Harrison, Fabio Petroni
Abstract:
The web is increasingly accessed by AI agents rather than humans. Every agent needs knowledge, especially in the life‑sciences, where agentic pipelines are growing fast. Access to the literature is a crucial part of that need, and resources such as Europe PMC, with over 40M indexed records, are widely used to meet it. Yet these resources were not built for AI agents: they take keywords and complex syntax and return whole papers, so every agent must learn the syntax, issue several searches, and read full papers to find the evidence it needs. We introduce EMBL AI Librarian, a knowledge layer that upgrades the Europe PMC interface for AI agents: an agent asks in natural language and receives evidence that answers it. A single LLM orchestrates the whole knowledge retrieval process: it plans complementary subqueries executed by the live Europe PMC search engine, then reads the selected papers and locates the relevant evidence. We evaluate Librarian across four benchmarks: literature synthesis, claim verification, open‑domain question answering, and downstream biology tasks such as protocol questions and sequence manipulation. On ScholarQABench, Librarian improves Citation F1 by more than 16 points over strong recently published baselines. Used as the retrieval layer of an existing claim‑verification pipeline, it increases agreement with expert consensus; and on the open‑form LitQA2 benchmark, a GPT‑5.4 agent scores about 8 points higher when grounded in Librarian than with web search. Overall, our results show that equipping life‑science agents with the Librarian knowledge layer improves performance across a range of tasks. We release our code publicly at https://github.com/petroni‑lab/librarian
Authors:Haoqing Wang, Xingrun Xing, Wei Xia, Ziheng Li, Yehui Tang
Abstract:
Agentic vision‑language models (VLMs), which interleave textual reasoning with explicit tool calls such as cropping and code‑based image manipulation, have emerged as a compelling paradigm for reliable and interpretable multimodal reasoning. However, recent studies have revealed that such models often use tools unfaithfully. Many process images are irrelevant to the question (e.g., the tool crops the wrong region or misses the queried target), yet the call still receives full credit and the model still answers correctly. Such decorative or misaligned tool calls waste computation and reveal that the model leans on prior knowledge or the original image rather than the evidence it retrieves. This may stem from two limitations of prevailing methods: the tool reward fails to distinguish useful from useless calls, and tool feedback carries no signal of usefulness. To this end, we introduce FaithEyes, a multi‑agent self‑judging framework. Concretely, we use a VLM to judge whether each process image helps answer the question. The judgement is injected into the reasoning context as part of the tool observation to help subsequent reasoning, and meanwhile is used to scale the tool reward by the helpful‑tool ratio to suppress reward hacking. To keep judgement available at evaluation and thus ensure train‑test consistency, we further design a multi‑agent framework where the model itself serves as a subagent to judge the tool calls from main agent, eliminating any dependence on an external model at inference. Training via a two‑stage SFT + RL pipeline on adapted open‑source data, FaithEyes attains competitive or superior accuracy across visual perception and reasoning benchmarks, while markedly improving tool faithfulness. The homepage is at https://github.com/Mosi‑AI/FaithEyes.
Authors:Hui Zhang, Julian Ferchow, Jie Song, Mirko Meboldt
Abstract:
Many dexterous manipulation tasks require the object to remain securely held throughout the interaction. From the perspective of hand‑object relational motion, such manipulation comprises four canonical skills: grasping, relocation, in‑hand rotation, and in‑hand translation. Human hands flexibly compose these skills to accomplish complex tasks. Existing approaches, however, model these skills separately with skill‑specific action constraints, objectives, or even dedicated hand morphologies, which breaks the compatibility and continuity required for long‑horizon composition. In this work, we present a unified framework that models all four skills in a single formulation that shares the same state and action spaces and a common objective structure. This formulation enables straightforward distillation of a single cross‑skill policy that performs strongly on every skill, generalizes to unseen objects, stays robust to disturbances, and chains skills seamlessly into long‑horizon manipulation. The framework also transfers effectively across different hand morphologies. Overall, our results suggest that different dexterous manipulation skills can be viewed as instantiations of a shared task formulation, revealing the intrinsic consistency across different behaviors.
Authors:Haiyang Wu, Weiliang Mu, Zhuofei Du, Dandan Zhong, Kaijie Shi, Haifeng Li, Chao Tao
Abstract:
Existing farmland remote sensing image (FRSI) segmentation follows a "Think with Intra‑Image" paradigm, assuming that the current image contains sufficient visual evidence for reliable segmentation. Yet farmland appearance varies with phenology and spatial context and is often confused with other land‑cover, making instantaneous, local observations inadequate. Thus, segmentation ambiguity stems not only from limited model representation, but more fundamentally from the required spatio‑temporal information lying beyond the current image. Based on this insight, we redefine FRSI segmentation from an information bottleneck perspective as a dynamic decision process driven by task‑relevant extra spatio‑temporal information gain. We further propose FarmSeeker, a dynamic FRSI segmentation agent that identifies ambiguous regions, reasons about their causes, and queries extra spatio‑temporal information on demand for accurate segmentation. To evaluate FarmSeeker, we construct GSFS‑Bench, the first global‑scale, high‑resolution FRSI segmentation benchmark that supports reasoning‑querying. Experiments show that FarmSeeker achieves more stable segmentation performance than existing methods. The project is publicly available at: https://withoutocean.github.io/FarmSeeker/
Authors:Zixuan Jiang, Binghao Qiang, Jiaying Chi, Yanqiao Zhu, Kai Yu, Xie Chen
Abstract:
Automatic speech recognition (ASR) has achieved substantial gains in transcription accuracy, yet verbatim transcription does not necessarily produce readily usable text. It retains fillers, repetitions, false starts, and self‑corrections that increase reading effort, obscure the speaker's final intent, and propagate unresolved or abandoned content to downstream tasks. Existing spoken‑to‑written methods process completed audio or transcripts but cannot revise emitted text when later speech changes how preceding content should be interpreted. We therefore formulate Agentic Speech Recognition (AgenticSR), an audio‑to‑clean‑text task that removes disfluencies, resolves self‑corrections, and normalizes written form while preserving the speaker's final intent. AgenticASR implements this task through an ASR‑‑Refiner architecture that repeatedly transforms a bounded active context and replaces its corresponding output span as audio arrives. This enables continual emission and revision over streams of arbitrary duration. We also introduce AASR‑Bench, a bilingual benchmark with fine‑grained atomic rubrics. Across multiple ASR front ends, AgenticASR attains the highest AASR‑Bench scores among evaluated systems. A human‑‑AI agreement study shows that rubric‑based judgments align with independent expert assessments. Ablations characterize Refiner capacity, context length, and the quality‑‑latency trade‑off between online and offline inference. Together, these results establish AgenticASR as a practical framework for intent‑preserving clean transcription during ongoing speech. Code, AASR‑Bench, and a demo will be released at https://github.com/AnXMuy/AgenticASR.
Authors:Chia-Ming Lee, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu
Abstract:
Diffusion language models (DLMs) expose a provisional prediction at every denoising step, creating an opportunity for generation‑time early exit that stops decoding before the schedule is exhausted. Existing early‑exit gates decide termination from fixed‑region confidence statistics or schedule‑dependent rules, evidence too coarse for a decision that freezes every remaining position at once, so they fire prematurely on long chain‑of‑thought outputs whose answers stabilize only near the end. Adaptive sampling, the other axis of training‑free acceleration, paces how quickly positions commit while decoding continues but never verifies that the output itself has stabilized. We introduce a training‑free, candidate‑aware early‑exit framework that keeps the two axes separate and matches each decision to evidence of its own scope. Confidence‑Verified Commit (CVC) governs when the sequence may stop by verifying confidence and sustained argmax stability over the dynamically extracted candidate span using a deterministic parser specified from each task's output format. Block‑Wise Early Commit (BWEC) governs where to accelerate by applying a cheaper local rule to non‑final blocks, while leaving the final block and global termination under CVC. We refer to their combination as LATCH (Localized Acceleration with Tracked‑Candidate Halting). Unlike prior methods, LATCH needs no suffix‑prompt construction; it is prompt‑anchor‑free but format‑aware. We evaluate LATCH end to end on 11 tasks under zero‑shot settings using LLaDA and Dream. LATCH stays within 2.0 percentage points of full‑decoding accuracy across all 22 evaluation settings, with one frozen hyperparameter set that transfers cross‑backbone untuned, while achieving end‑to‑end TPS speedups of 9.3‑17.8x on short‑answer tasks and 2.0‑3.3x on long‑reasoning tasks.
Authors:Mingxiao Liu, Yitong Li, Haoren Zhao, Yaoxiang Bian, Jianan Ma, Jian Zhang, Jialuo Chen, Xinhao Deng, Zhen Wang
Abstract:
Large Language Model (LLM)‑driven multimodal agents are increasingly deployed to execute autonomous tasks via continuous audio interaction. While this paradigm enhances interaction naturalness, it introduces a critical yet under‑explored attack surface, as audio inputs inevitably contain environmental noise beyond user control. In this paper, we investigate concurrent audio prompt injection attacks targeting multimodal agents. Distinct from traditional acoustic attacks on voice devices, we propose novel techniques for instruction augmentation and scenario concealment. These methods allow malicious audio instructions to imperceptibly "piggyback" onto user speech, thereby hijacking agents to execute malicious actions. To systematically quantify this threat, we construct AudioAgentSecurity, the first comprehensive benchmark for audio instruction injection attacks, encompassing 8 real‑world task scenarios and 10 distinct attack patterns. We evaluate 11 state‑of‑the‑art agents, including Gemini 3 Pro and GPT‑4o‑audio. Notably, our methods achieve an average Attack Success Rate (ASR) of 69.10% against the advanced Gemini 3 Pro. To counter this threat, we further introduce Cascaded Audio Decoupling and Verification (CADV), a defense mechanism based on source separation and consistency analysis. Compared with existing prompt‑level defenses, CADV achieving up to 96% detection accuracy and providing effective protection against a broad range of acoustic injection attacks. Finally, real‑world experiments with human volunteers on Doubao AI Smartphone in diverse dynamic real‑world scenarios confirm the attacks' high stealth and efficacy, while demonstrating that our defense reliably mitigates these vulnerabilities.
Authors:Hail Song, Seokhwan Yang, Jiwon Yang, Woojin Cho, Woontack Woo
Abstract:
We propose S‑Avatar, a novel method for generating photorealistic 3D head avatars from a single image using a diffusion‑guided 3D model generation module and strategies for animating 3D Gaussian Splatting (3DGS). While single‑image head avatar reconstruction is crucial for lifelike Virtual Reality (VR) applications, existing approaches often struggle to preserve 3D consistency under unseen viewpoints. S‑Avatar addresses this limitation through a three‑stage pipeline. First, a high‑resolution 3DGS is synthesized directly from a single image using a diffusion‑based Gaussian splat generation module. Next, the parametric head model FLAME is aligned with the generated 3DGS by optimizing its parameters and spatial transformations. Finally, to adapt the 3DGS to FLAME variations, we construct a binding template that encodes the spatial relationship between the initial splats and FLAME. The dynamic 3D head avatar can then be rendered in real time by deforming the 3DGS with the binding template. By combining diffusion‑guided canonical 3DGS generation with FLAME‑based control, our method achieves efficient and accurate reconstruction with enhanced 3D consistency. Evaluations on public datasets demonstrate that S‑Avatar outperforms state‑of‑the‑art methods in novel‑view and expression generation, achieving superior realism and consistency. Consequently, our approach represents a significant advance in accessible avatar creation, applicable to a wide range of VR/AR applications. The project page is available at https://github.com/hailsong/savatar.
Authors:Rui Xu, Yunke Wang, Linwei Tao, Wenjie Xuan, Yong Luo
Abstract:
AI‑assisted research ideation has emerged as a promising paradigm for accelerating scientific discovery, with systems now capable of generating research directions conditioned on papers, topics, or lightweight researcher contexts. Yet current systems largely optimize individual suggestions in isolation. This leaves two blind spots. First, coarse researcher representations may elicit mainstream directions that appear broadly feasible, but lack sufficient researcher‑specific grounding. Second, independent recommendations can concentrate a community's portfolio around recurring high‑probability themes. To address these blind spots, we propose DivAlign, a four‑stage pipeline for alignment‑preserving de‑homogenization. DivAlign extracts fine‑grained researcher profiles, generates profile‑conditioned candidate directions, scores them along three alignment dimensions (Executability, Comprehensibility, and Growth Potential), and surfaces researcher‑local directions while reducing redundancy across the community portfolio. On a benchmark we construct from 95 AI researchers across five subfields, DivAlign reduces community‑level redundancy while preserving researcher‑direction fit. Compared with coarse single‑shot ideation, it lowers average pairwise similarity from 0.331 to 0.294 and nearest‑neighbor similarity from 0.704 to 0.608. Compared with the independent top‑choice variant, DivAlign reduces nearest‑neighbor similarity from 0.663 to 0.608 while retaining 99.9% of the researcher‑direction fit score. Code and data are available at https://github.com/Ruixxxx/DivAlign.
Authors:Shuang Liang, Haoyang Zhou, Yifan Gong, Guowei Wang, Xiting Wang
Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre‑rollout prompt selection can reduce this waste by screening prompts before rollout generation. However, existing pre‑rollout methods struggle to balance exploitation and exploration: repeatedly exploiting historically informative prompts can narrow training coverage, whereas broader exploration can lower the fraction of informative prompts. To address these limitations, we introduce LEEPS, a Latent‑Guided Explore‑‑Exploit Prompt Sampler that adaptively balances the reuse of previously observed informative prompts with continued exploration of uncertain ones. LEEPS partitions candidates into exploit and explore portfolios and adaptively allocates rollout budget according to their recent non‑trivial ratios. It further uses representation‑space neighbors and historical rollout outcomes to prioritize uncertain prompts likely to yield non‑zero reward variance, thereby making exploration more targeted without additional rollouts. Across six mathematical reasoning benchmarks, LEEPS achieves the highest average score at both model scales, with relative gains of 2.6% and 3.7% over the strongest baseline for Qwen2.5‑Math‑1.5B and 7B, respectively, and generally improves faster during the training process. It also achieves the highest average score across the three evaluated OOD general‑reasoning benchmarks at both model scales and adds only about 2 seconds of online sampling overhead per training step. Code is available at https://github.com/ShuangLiangX/LEEPS.
Authors:Yiming Xu, Jihua Kang, Chunsai Du, Qifan Zhang, Wangqiu Zhou, Yiting Wu, Tianqi Li, Qi Song
Abstract:
In demanding professional environments and meeting review scenarios, lengthy text often imposes a high cognitive load. To facilitate efficient information communication, transforming verbose text into logically clear diagrams is essential. Scalable Vector Graphics (SVG) provide an effective representation for this purpose due to their editability and resolution independence. However, current research on Text‑to‑SVG generation remains hindered by three major challenges: (1) the scarcity of datasets for complex, logic‑rich diagrams; (2) the absence of explicit layout priors, which leads to chaotic spatial arrangements; and (3) the lack of fine‑grained visual feedback to validate rendered outputs and correct aesthetic defects. To address these challenges, at the data level, we introduce DocMeetSVG‑100K, a large‑scale SVG dataset tailored for document authoring and meeting review scenarios. At the model level, we propose GVR‑Coder, a novel framework designed to generate high‑quality logical diagrams from lengthy professional texts. Specifically, we adopt a curriculum‑driven rejection sampling fine‑tuning to progressively enhance the model's capability in modeling complex structures, while explicitly incorporating layout constraint knowledge during training. In addition, we introduce reinforcement learning from dual rendering feedback, a mechanism that provides implicit feedback through reward signals to jointly optimize structural complexity and visual aesthetics. Furthermore, we design a generate‑verify‑repair agent loop, which improves generation quality through explicit, fine‑grained feedback and targeted refinement. Extensive experiments demonstrate that GVR‑Coder outperforms competitive baselines and reliably produces logically coherent and visually appealing diagrams. Code and data are available at https://github.com/CurryaNa/GVR‑Coder.
Authors:Yabin Xu, Fangtao Zhang, Fan Wang, Zhan Wang, Honghua Chen, Mingqiang Wei, Haoran Xie, Sam Kwong
Abstract:
Wind turbine blade defect detection remains highly challenging in real‑world inspection scenarios due to limited on‑site data and the subtle visual characteristics of defects. In practice, blade defects are often small‑scale, low‑contrast, and difficult to distinguish from complex backgrounds, which significantly limits the robustness of existing detectors. To address these challenges, we propose BladeYOLO, a defect detection framework for wind turbine blades. Specifically, we integrate a Vision Transformer (ViT) backbone initialized with DINOv3 self‑supervised pre‑trained weights into YOLOv12‑L, enabling the transfer of large‑scale generic visual priors to blade defect detection and improving feature representation under limited training annotations. To enhance the perception of subtle defects, we further develop a Mamba‑guided Weak‑Defect Enhancement module, which consists of a Detail‑Enhanced Multi‑scale Branch for preserving high‑frequency structural cues and a Cross‑Mamba module for progressively propagating high‑level semantic guidance to shallow features. In addition, we introduce a lightweight Style‑Injector module that captures environment‑related style information via Fourier decomposition and injects it into selected ViT self‑attention layers, thereby improving robustness against environment‑induced appearance variations. Extensive experiments demonstrate that BladeYOLO achieves superior performance on the WTBlade‑Defect dataset, with additional annotation‑budget experiments showing its favorable performance under reduced training annotations. Evaluation on the public Wind Surface Defect dataset further provides supportive evidence for the cross‑dataset robustness of BladeYOLO. In particular, on this public dataset, BladeYOLO outperforms the best competing method by 3.5% in mAP_50 and 2.5% in mAP_50‑95.
Authors:Henglin Liu, Fangyuan Kong, Jing Wang, Yizhou Lin, Nisha Huang, Chang Liu, Xintao Wang, Pengfei Wan, Kun Gai, Xiu Li
Abstract:
Recent advances in preference alignment for diffusion‑based video generation, particularly via Direct Preference Optimization (DPO), have significantly improved visual quality. However, temporally sparse artifacts such as motion collapse, object flickering, and color oversaturation remain a major barrier to perceptual realism. Existing methods struggle with these issues due to two key limitations: (1) the preference attribution bottleneck, where offline human annotations are costly and fail to accurately capture learning dynamics, while online reward signals are rollout‑aware but often unstable and biased; and (2) temporal credit misallocation, where uniformly applied supervision cannot effectively target the brief segments in which artifacts occur. To address these challenges, we propose concentrated Implicit Preference Optimization (cIPO), a post‑training framework for video diffusion models. cIPO derives implicit preference signals directly from the denoising process: given a real video, the model adds forward noise and reconstructs it via iterative denoising, treating the original as the preferred sample and the reconstruction as the dispreferred one. This formulation captures inference‑time errors without requiring human annotations or external reward models. Moreover, frame‑level discrepancies between original and reconstructed videos reveal when failures occur. cIPO leverages this by computing temporal reconstruction errors and concentrating optimization on high‑error segments, enabling more precise correction of failure‑prone regions. Extensive experiments demonstrate that cIPO consistently enhances video authenticity and temporal coherence across multiple datasets, highlighting the effectiveness and efficiency of implicit preference with temporally concentrated optimization.
Authors:Sangwoo Jung, Dongjae Lee, Chiyun Noh, Ayoung Kim
Abstract:
Recent advances in 4D radar enable robust perception in adverse weather; however, the inherent sparsity, noise, and limited positional precision of radar point clouds pose significant challenges for registration‑based odometry. In this letter, we propose RaDiVe, a 4D radar odometry framework designed to improve the accuracy and robustness of radar point‑cloud registration. We introduce a distance‑bounded Normal Distributions Transform (NDT), which improves optimization stability and computational efficiency by restricting the correspondence search to near‑distance voxel pairs. To mitigate measurement ambiguity, we propose a velocity‑discrepancy point uncertainty model that weights each input 4D radar point according to the discrepancy between its measured Doppler radial velocity and the radial velocity predicted from the estimated ego‑velocity. Furthermore, we incorporate Signed Distance Function (SDF)‑based surface point extraction via implicit neural mapping to construct a geometrically consistent and noise‑filtered local submap. Evaluations across multiple public datasets demonstrate that RaDiVe outperforms existing 4D radar odometry baselines by 44.4% in translational Absolute Trajectory Error (ATE) and 21.3% in rotational ATE on average, while maintaining real‑time performance. The source code will be made publicly available to the robotics community: https://github.com/to‑be‑open‑sourced.
Authors:Debin Meng, Jiaming Yang, Zefang Zong, Tengyue Xu, Haining Xie, Yang Li, Peng Chen
Abstract:
Large language models (LLMs) and LLM‑based agents are increasingly being deployed to automate complex workflows, promising to revolutionize data management and processing. However, existing benchmarks predominantly focus on simplified Text‑to‑SQL translation or data analysis, leaving the critical and complex domain of end‑to‑end data engineering largely unexplored. To bridge this gap, we introduce DataClawEval, the first comprehensive benchmark designed specifically to evaluate the end‑to‑end task completion capabilities of autonomous agents in real‑world data engineering scenarios. Built upon production‑grade code authored by professional enterprise data engineers, it comprises 100 rigorous, end‑to‑end tasks spanning five execution engines: PySpark, MySQL, HiveSQL, PrestoSQL/Trino, and FlinkSQL. Rather than non‑deterministic LLM‑as‑a‑judge scoring, each task is executed within a case‑specific, isolated sandbox and graded by deterministic, rule‑based scripts. Evaluating 16 frontier agents exposes critical limitations: The strongest model attains only 74.9 overall, and no single model dominates, as each excels on a different engine, revealing strict domain specialization rather than omnipotent proficiency. Thus, autonomous data engineering remains a formidable, unresolved challenge. We release our dataset, containerized environments, and deterministic evaluation scripts at https://github.com/Dicemy/DataClawEval/tree/master
Authors:Sweta Banerjee, Alireza Teimoury, Nils Porsche, Alexandra K. Stoll, Viktoria Weiss, Niklas Hargarter, Jonas Ammeling, Thomas Conrad, Christoph Stroblberger, Christopher Kaltnecker, Robert Klopfleisch, Christof A. Bertram, Katharina Breininger, Marc Aubreville
Abstract:
Pathology foundation models (FMs) are models trained on vast amounts of typically unlabeled data and have been shown to yield regularized latent spaces that can be used effectively in downstream classification tasks. This is also true for the classification of mitotic figures vs. other cells. However, it is so far unclear if the latent space of current FMs provides features that are discriminant and spatially suitably resolved to also serve as a backbone for dense object detection paradigms. In this work, we investigate this question for common current pathology FMs (UNI, UNI2‑h, Virchow, Virchow2, H‑optimus‑0, H‑optimus‑1) and compare their performance against a fully end‑to‑end trained baseline based on a ResNet50 architecture. We combine FM backbones with representatives of single stage, dual stage and self‑attention‑based detectors (RetinaNet, Faster R‑CNN, Deformable DETR respectively) on the multi‑domain MIDOG++ dataset, and on the TUPAC16 dataset as an out‑of‑domain case. We show that the H‑optimus‑0 and Virchow models yielded competitive performance, indicating that the latent spaces of current FMs, all trained on image‑level self‑supervision, are suitable for direct mitotic figure detection and may be slightly more robust on our out‑of‑domain test case. All code is made available publicly at https://github.com/DeepMicroscopy/FM4MFdet.
Authors:Florian Fervers, Sebastian Bullinger, Christoph Bodensteiner, Michael Arens
Abstract:
Tensor operations represent a cornerstone of modern scientific computing. However, the Numpy‑like notation adopted by predominant tensor frameworks is often difficult to read and write and prone to so‑called shape errors, i.a., due to following inconsistent rules across a large, complex collection of operations. Alternatives like einsum and einops have gained popularity, but are inherently restricted to few operations and lack the generality required for a universal model of tensor programming. To derive a better paradigm, we revisit vectorization as a function for transforming tensor operations, and use it to both lift lower‑order operations to higher‑order operations, and conceptually decompose higher‑order operations to lower‑order operations and their vectorization. Building on the universal nature of vectorization, we introduce einx, a universal notation for tensor operations. It uses declarative, pointful expressions that are defined by analogy with loop notation and represent the vectorization of tensor operations. The notation reduces the large APIs of existing frameworks to a small set of elementary operations, applies consistent rules across all operations, and enables a clean, readable and writable representation in code. We provide an implementation of einx that is embedded in Python and integrates seamlessly with existing tensor frameworks: https://github.com/fferflo/einx
Authors:Haotian Zhang, Hao Chen, Han Guo, Zhengxia Zou, Zhenwei Shi
Abstract:
Despite substantial progress in remote sensing multi‑temporal change detection (MTCD), most existing MTCD methods still represent the dynamic process at each spatial location over the entire observation period using a single change category associated with the final observation. This implicit single‑change assumption limits their ability to characterize regions of recurrent change closely related to human activities. To address this limitation, we introduce Urban Building Dynamics Detection (UBDD), which identifies building‑change dynamic footprints, i.e., the temporal intervals in which changes occur, from multi‑temporal imagery and produces pixel‑wise classification masks. For regions undergoing two or more changes, UBDD introduces an independent multi‑change class for unified representation, thereby enabling unified modeling of single‑ and multi‑change processes. Furthermore, we propose FootprintNet, which abstracts building‑change processes as interactions between latent states and actions, and imposes state‑action transition constraints to guide the learning of causally coherent change trajectories. It further exploits temporal change‑boundary cues to enhance feature contrast across boundary sides, thereby improving the discrimination among different dynamic footprints and enabling accurate detection of dynamic footprints. Moreover, we introduce the Building Change Dynamics Score (BCDS) to address the inability of conventional metrics to reflect the temporal proximity between predicted footprints and labels. It evaluates predictions according to their preservation of change semantics and temporal offsets from the corresponding labels. Extensive experiments on TSCD, MUDS, and WUSU demonstrate that FootprintNet outperforms current state‑of‑the‑art methods. The code is available at https://github.com/zmoka‑zht/FootprintNet.
Authors:Ziyang Rao, Yiren Zhao, Weiyu Guo, Ben Fei, Yandong Guo, Hui Xiong
Abstract:
Flow matching (FM) has become a popular action head paradigm for modern embodied models. However, as a conditional generative model, it does not explicitly expose its inherent uncertainty, producing faulty action chunks even when it misinterprets the scene or encounters out‑of‑distribution (OOD) inputs. Therefore, determining when an FM‑generated action can be trusted is essential for safe deployment, yet existing uncertainty estimation methods on real‑time control suffer from several issues: extra training budget, high computational overhead, and low generalization ability. In this work, we provide a geometric interpretation of FM uncertainty in the velocity field, showing that uncertainty manifests as deviation from an ideal affine‑isotropic contraction field. Building on this observation, we introduce denoising acceleration (\mathrmaccel), a highly‑generalizable and cost‑free uncertainty proxy that measures the bending of the denoising trajectory from a single forward pass, without additional model evaluations, training, or resampling. We theoretically and empirically demonstrate that \mathrmaccel is a faithful proxy for FM uncertainty and further test its utility in online failure detection. Results show that \mathrmaccel identifies failing rollouts well before termination, matching or even outperforming costly resampling‑ and training‑based baselines across settings under realistic deployment budget. Code and demos available at: https://github.com/rrrrrrzy/fm‑geometry.
Authors:Dongxiu Liu, Haoyi Niu, Peng Cheng, Yuan Gao, Xirui Kang, Sangli Teng, Koushil Sreenath, Xianyuan Zhan
Abstract:
In the physical world we inhabit, space and time are fundamentally continuous. However, existing machine learning paradigms for world modeling are largely confined to discrete‑time prediction, thereby exhibiting significant inefficiency in capturing the dynamics of physical world. We introduce Physical‑Time Flow (PT‑Flow), a novel approach that learns a continuous latent velocity field operating in physical time. Crucially, the underlying dynamics of sequential data are parameterized by an ordinary differential equation (ODE) embedded in a well‑structured representation space. Under this paradigm, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space. Building upon PT‑Flow, we construct ODEWorld, a continuous‑time latent world model that is both efficient and versatile. By extracting time‑variant features and enforcing ODE properties on both the dynamical representation space and the latent velocity field, ODEWorld effectively addresses the long‑standing representation collapse issue in latent world model literature. This also enables high‑quality image reconstruction even after long‑horizon prediction. Moreover, its continuous nature allows for arbitrary temporal resolution and even backward prediction, which is impossible for most discrete‑time models. Lastly, ODEWorld can provide rich planning‑oriented information to facilitate downstream policy learning. Comprehensive experiments demonstrate that ODEWorld successfully reconciles planning‑conducive dynamics abstraction with visual realism, excelling in both video generation and robotic control. \hrefhttps://dstate.github.io/odeworld_website/Project Website.
Authors:Zaiyan Zhang, Qiangqiang Yuan, Jie Li, Ziyang Lihe, Yu Wan, Yuzeng Chen, Xin Su, Liangpei Zhang
Abstract:
Remote sensing images acquired by unmanned aerial vehicles (UAVs) and satellites are often degraded by adverse weather, illumination variation, and imaging artifacts, which may co‑occur and jointly induce global distribution shifts and local structural corruption. Although All‑in‑One image restoration offers an appealing unified alternative to task‑specific pipelines, existing methods still suffer from weak or implicit degradation cues and parameter redundancy caused by full‑rank multi‑expert designs with overlapping restoration behaviors. We propose CoRE‑UIR (Common and Residual Experts for Universal Image Restoration), a prior‑guided global‑local framework centered on the Common‑and‑Residual Expert Block (CoRE). CoRE explicitly decomposes restoration capacity into a common dense expert for degradation‑invariant restoration and low‑rank residual experts for degradation‑specific compensation, enabling adaptive specialization without redundant expert replication. Built on this design, Degradation Prior Embedding (DPE) adapts frozen CLIP features into an explicit restoration‑oriented prior, while Global Feature Modulation (GFM) aligns global feature statistics before local residual compensation. We also construct MDVD‑108K (Multi‑Degradation VisDrone), a large‑scale UAV restoration dataset covering both single and compound degradations, together with a real‑world test set. Extensive experiments on multiple datasets show that CoRE‑UIR improves the overall average PSNR by 1.05 dB while running 11.83× faster and reducing peak memory by 85.3% relative to the strongest baseline, BaryIR, thereby maintaining a favorable quality‑efficiency trade‑off. Evaluations on downstream tasks and unseen degradation also validate the generalizability of CoRE‑UIR. The code and dataset will be released at https://github.com/zzaiyan/CoRE‑UIR.
Authors:Yan Kong
Abstract:
Temporal graph learning is commonly organized around the evolution of node states or the encoding of interaction histories. We study an underexplored, operator‑centric question: should the graph propagation mechanism itself evolve over time? We introduce Dynamic Spectral Filtering (DSF), which represents propagation at snapshot t by a Chebyshev polynomial filter with vector‑valued, time‑dependent coefficients. DSF explicitly treats these compact multi‑order coefficients as recurrent temporal states. A recurrent branch proposes updates, while multiplicative global and order‑specific gates regulate their magnitude. The temporal state is independent of the number of nodes. On MOOC, Wikipedia, and Reddit temporal link‑prediction benchmarks, converged DSF runs attain AP scores of 0.7851, 0.9088, and 0.9860, respectively, with 93K to 133K trainable parameters, 68 to 182 MB peak GPU memory, and 1.6 to 2.1 seconds of training per epoch. Against the closely related DEFT baseline, DSF is better on MOOC, within 0.001 AP on Reddit, and modestly lower on Wikipedia, while using 8.3 to 8.6 times fewer parameters, 25 to 33 times less GPU memory, and 5 to 19 times less time per epoch. Relative to all measured alternatives, it uses 3.3 to 38.6 times less GPU memory. These results support direct spectral‑response evolution as a useful temporal inductive bias when computational efficiency is a first‑class requirement.
Authors:Woo Chul Shin, Zhenyang Chen, Alfred Cueva, Nadun Ranawaka Arachchige, Yingyan Celine Lin, Benjamin Joffe, Shreyas Kousik, Danfei Xu
Abstract:
Visuomotor policies have advanced on manipulation tasks where the target object stays static during execution, but real deployments break this assumption: parts drift on conveyors and fruits sway in the wind. We introduce Static In, Dynamic Out (SIDO), a counterfactual action augmentation that enables a policy trained only on static object demonstrations to adapt to unseen object motion at test time. Our key idea is to factorize moving object manipulation into two sub‑problems: predicting where the object will be, and reaching that predicted pose. SIDO displaces the object to a counterfactual future position and morphs the demonstrated action chunk to preserve the hand‑object relative pose, yielding a goal‑conditioned policy. At deployment an object pose predictor supplies the future position. Across three simulated tasks (Mug, Square, Stack) under five object motion patterns and two real‑world tasks (Gantry, Peachtree), SIDO improves moving object success over the baselines while preserving static object performance. Project website: https://sido‑staticindynamicout.github.io/.
Authors:Yanyu Ren, Yunfeng Bai, Xizheng Wang, Li Chen, Dan Li
Abstract:
Multi‑agent vibe coding promises to accelerate software development, yet existing benchmarks rely on synthetic environments that ignore practical time and monetary costs, conflate reasoning with communication, and reward only superficial completion. We introduce multi‑agent from‑scratch evaluation benchmark, MSEval, evaluating multi‑agent coding on real‑world tasks. Grounded in 10 authentic, full‑stack projects across 10 domains, MSEval scores performance using hierarchical requirements and deterministic rubrics. Its execution engine, LegoGent, tests 10 collaboration topologies where agents coordinate via periodic sync intervals and deploy through native CI/CD pipelines. Concurrently, the automated grader TAgent dynamically probes implementations to jointly measure functional success, latency, and prefix‑cached token cost. Across 100 runs, MSEval reveals that organizational topology rivals model capability in shaping the speed‑‑cost‑‑quality trade‑off. For identical tasks and models, varying the topology shifts scores by over 30 points and doubles wall‑clock time. Structured pipelines converge fastest with the highest quality, whereas heavy managerial oversight degrades performance. Ultimately, MSEval establishes a rigorous, reproducible standard for measuring how multi‑agent teams actually build software. The benchmark is released at https://github.com/robinren03/MSEval.
Authors:Gerrit Großmann, David A. Selby, Sebastian J. Vollmer
Abstract:
Structural causal models are the standard language for reasoning about interventions and counterfactuals, but they describe static variables, typically measured once, and usually forbid cyclic dependencies. Many systems we care about, such as patients, climates, and economies, instead evolve continuously in time, are observed at irregular time points, and contain feedback loops. We argue that neural operator learning provides a natural foundation for causal reasoning in this setting, and propose Orca, a framework in which each node of the causal graph is a function of time and each mechanism is a learned map between function spaces. We extend existing neural operator architectures to express causal mechanisms: a mechanism computes the function value of a node from its parent nodes by taking several parent functions as input, respects the arrow of time, and treats latent exogenous noise as a function that can be inferred and reused for counterfactuals. We formalize the model class and demonstrate counterfactual reasoning on synthetic continuous‑time examples. Code is available at https://github.com/gerritgr/orca
Authors:Kaifan Zhang, Lihuo He, Yuqi Ji, Junjie Ke, Lukun Wu, Tianhao You, Xinbo Gao
Abstract:
Recent EEG‑to‑image retrieval models have achieved strong performance in identifying viewed images from semantically diverse candidates. Yet such success does not reveal what visual information supports the match. A model may readily identify a cheetah among tools, plants, and vehicles, but can it still distinguish the viewed cheetah from the same scene with the cheetah replaced by a dog? Motivated by this question, we introduce EEG‑EditBench, a diagnostic benchmark that examines this question through controlled edits of object identity, attributes, background, and object presence. Built from the 200 THINGS‑EEG2 test images, EEG‑EditBench contains 2,137 quality‑controlled edits and evaluates eight representative EEG visual decoding models. Our results show that strong standard retrieval does not consistently transfer to edit‑based evaluation, with fine‑grained attribute changes presenting the greatest challenge. EEG‑EditBench reveals model behavior hidden by aggregate retrieval accuracy and provides a controlled basis for studying what visual information EEG‑image models preserve. The code and complete dataset are publicly available.
Authors:Lorenzo Sciandra, Samuele Fonio, Roberto Esposito
Abstract:
Understanding and exploiting the training dynamics of overparameterized deep neural networks remains a central challenge in modern machine learning. Recent evidence on Neural Collapse (NC) shows that class representations and classifiers exhibit highly structured geometry, while the Tunnel Effect suggests that only a subset of layers is essential for feature extraction. We combine these two perspectives and propose an NC‑inspired training framework for simplifying deep networks during training. Our method monitors representation dynamics through the Inverse Fisher Criterion, a stable and efficient proxy for the variability collapse behavior, to identify both the split point between feature extraction and classification and the training stage at which simplification becomes viable. We then replace the trailing layers with a lightweight classification head and continue training the reduced model. Experiments on image‑classification benchmarks across MLP, VGG, and ResNet architectures show that the proposed method achieves substantial parameter reductions while maintaining accuracy comparable to that of the full model. Code to reproduce the experiments can be found at: https://github.com/LorenzoSciandra/NNS.
Authors:Yijia Xiao, Rujun Han, Yanfei Chen, Zifeng Wang, Ke Jiang, Zhongying CuiZhu, Vishy Tirumalashetty, Wei Wang, Burak Gokturk, Tomas Pfister, Chen-Yu Lee
Abstract:
Powered by advances in LLMs and autonomous agents, deep research has become one of the most widely adopted agentic products. However, most deep research systems write general‑purpose reports, which are inadequate for financial deep research. Financial research demands specialized knowledge to analyze historical patterns and forecast upcoming events. Automating financial deep research therefore requires both a layered harness to drive the research agent and a verifiable, point‑in‑time benchmark that prevents leakage of future information. We present FinanceHarness, a harness that runs finance‑oriented tools and practitioner‑guided workflows, automating financial deep research end to end: environment and data construction, the agent execution loop, and reward modeling. We further propose FinanceGym, comprising thesis‑driven research questions and rubrics that combine pre‑cutoff and post‑cutoff criteria. Professional expert validation yields an 82% pass rate. Even leading LLMs and agents score below 40% on the rubrics, showing that FinanceGym is challenging and leaves substantial headroom. With the same open‑weight backbone, FinanceHarness improves the overall rubric score from 25.3% to 32.4%. FinanceHarness is available at https://github.com/Yijia‑Xiao/FinanceHarness.
Authors:Christian Rosenthal
Abstract:
An open‑weight LLM can write composition setpoints every five minutes. What a plant still needs is a hard check: named constraints, logged margins, and an admit/block decision before the regulatory layer moves. This paper puts that check in a rule‑based forked‑twin counterfactual gate (nine pinned constraints) and leaves the regulatory layer unchanged. On Skogestad's Column A the ladder is PID‑only (C0), linear MPC (C1), ungated agent (C2), and gated agent (C3) under one contract: identical level closure (M_D, M_B), scenarios, and seeds; C2/C3 share the linear‑MPC backend. The split is not subtle. Off‑nominal target acquisition: the agent beats Pareto‑tuned linear MPC in the strong band (C2/C1 IAE ratio 0.361 at the upper CI). Disturbance rejection on the same 16‑point grid inverts by 16.03 at the upper CI (10.18 at the point estimate), where an ungated LLM supervisor does not belong. The gate compresses a specification‑abandonment attractor into a bounded offset (d approx. ‑1.4; P95 cell IAE 11.5 to 0.77). A one‑line prompt fix removes the attractor at source (6/10 to 0/10; sensitivity only, not a new headline). In a 250‑cell statistical pass, 534 of 590 gate interventions are spec‑on‑bound geometry: the operating specification sits on a safety limit, so a well‑behaved OP becomes inoperable while misbehaving ones are only contained; 318 blocks still correct actively harmful proposals. Headlines are single‑column and model‑conditional on DeepSeek‑V4‑Flash. A second‑family sweep (NVIDIA Nemotron‑3‑Super) keeps the disturbance‑rejection fails band and plant‑side failure geography; magnitudes and protocol operability stay model‑conditional, and Super target‑acquisition strong cells are survivors only (not confirmation). Transfer means twin, constraint envelope, and setpoint interface, not a second plant class measured here.
Authors:Lukas Kubelka, Alexander Bott, Frank Döhner, Saksham Kiroriwal, Georg Zeeb, Julia Butte, Julius Pfrommer, Jürgen Beyerer, Tobias Käfer
Abstract:
We propose the Virtual Process Dossier (VPD), a Knowledge Graph‑based data catalogue that also captures workflow provenance. We developed VPD for multi‑stage manufacturing use‑cases where downstream AI‑based optimization tasks require to distinct between datasets generated during individual workflow steps. VPD provides these datasets in a FAIR manner and makes both prospective and retrospective workflow provenance explicit. Our contributions are: (1) the VPD ontology that serves as the catalogue's semantic core; (2) the VPD provenance framework that integrates ontology instantiation into the production environment; and (3) the VPD user interface that provides human‑centered interaction with the VPD Knowledge Graph. The ontology and code are available at https://github.com/kubeluk/VirtualProcessDossier .
Authors:Enric Gusó, Xavier Serra
Abstract:
Most Music Source Separation (MSS) models do not generalize well to live music recordings because they are trained on studio recordings alone, disregarding the venue acoustics, the speaker system's response and audience noise. We propose to bridge this gap by providing and training a model on two novel datasets. First, we present CrowdioSet: a noise dataset comprising 4800 real ambience tracks from Freesound and synthetic sing‑alongs for the vocals in MUSDB18 and MOISESDB datasets, generated from zero‑shot singing voice conversions. CrowdioSet enables effective audio denoising for live recordings, resulting in superior separation both in objective and subjective evaluations. Second, we introduce PaRIRset, a stereo impulse response dataset captured across 40 professional concert venues using a microphone array. Our results show that adding PaRIRset RIRs increases the performance of a MSS model compared to using real RIRs from Speech Enhancement tasks alone. We make the examples, code, model weights, PaRIRset, and CrowdioSet freely available to the public.
Authors:Yuhang Zhu, Mingxuan Du, Benfeng Xu, Jie Gao, Lingyun Yu, Hongtao Xie
Abstract:
Role‑playing agents (RPAs) have become one of the most important consumer applications of large language models. Users engage in multi‑turn conversations with RPAs for experiences such as emotional comfort, making reliable evaluation essential for measuring capability, comparing systems, and guiding further improvement. Existing benchmarks, however, typically require an RPA to continue a fixed dialogue history and then evaluate the continuation using a fixed rubric detached from the user. We identify and empirically demonstrate two limitations of this design. First, an RPA's output is shaped by the preceding dialogue history, preventing a scientifically grounded assessment of its role‑playing ability in real multi‑turn settings. Second, user experience varies substantially across individuals, and conventional fixed rubrics need not align with user satisfaction. We therefore introduce PALATE (Person‑Aligned LLM‑Simulated‑User Assessment with Tailored Evaluation), a scalable RPA benchmark built on user simulators. PALATE is accompanied by a pool of 300 character profiles. Its main evaluation trains five per‑user simulators and lets them engage candidate RPAs in free‑form, multi‑turn conversations over a pre‑frozen panel of character profiles. Alongside a general quality rubric, we construct personalized rubrics to measure user satisfaction; on held‑out annotated data, the personalized rubrics show higher agreement with human judgments than the general rubric. In the main evaluation of 16 candidates, PALATE separately characterizes generic turn quality, long‑horizon session capability, and per‑user experience on multi‑turn trajectories co‑constructed by each candidate. It thereby produces interpretable evaluations of specific user‑RPA pairs rather than compressing systems into a single user‑independent ranking.
Authors:Zhilin Wu, Zhangkai Ni, Chengmei Yang, Longzhen Yang, Yihang Liu, Ying Wen, Lianghua He
Abstract:
In clinical practice, patients often undergo multiple imaging examinations over successive visits, yielding longitudinal data. Modeling such temporal information is crucial for reliable assessment of disease progression and treatment response. However, despite the rapid advancement of multimodal large language models (MLLMs), longitudinal medical visual reasoning remains largely underexplored. To fill this gap, we propose LoMeVQA, a comprehensive benchmark consisting of 206K longitudinal visual question answering (VQA) pairs for temporal medical image analysis. LoMeVQA covers five tasks: progress classification, progress description, progress report generation, differential region grounding, and differential region description. To construct the dataset, we develop an automated pipeline that (1) organizes patient records chronologically, (2) extracts clinically meaningful entities via a medical knowledge graph, and (3) models their temporal evolution to guide large language models in generating high‑quality longitudinal VQA pairs. Extensive evaluations demonstrate that both general‑purpose and medical‑domain MLLMs perform poorly on LoMeVQA, revealing substantial limitations in temporal reasoning. To address these limitations, we introduce MedLong‑8B, which achieves state‑of‑the‑art performance across all tasks. Beyond benchmarking, we conduct detailed analyses that uncover key failure modes and shed light on how to improve longitudinal medical visual reasoning. Our data is available at: https://github.com/pepperbubble/LoMeVQA
Authors:Wei Chen, Junkai Li, Tongguan Wang, Hui Liu, Feiyue Xue, Chuanxiang Ma, Ying Sha
Abstract:
Multimodal Sentiment Analysis (MSA) aims to interpret complex human emotions by integrating natural language with non‑verbal modalities. Non‑verbal modalities share a structural isomorphism with natural language, as both can be viewed as feature sequences evolving over time. This isomorphism enables the transformation of non‑verbal modalities into text‑like tokens for unified semantic reasoning. Large Language Models (LLMs), designed to understand and generate sequential data, can thus be utilized to interpret complex affective sequences. However, existing LLM‑based methods primarily capture low‑level superficial features, failing to model affective semantics arising from structural variations and contextual interactions. To address this limitation, we propose SentiLLM, a unified framework that leverages Semantic‑Aligned Structural Abstraction to distill continuous raw signals into compact, semantically meaningful tokens. Specifically, we introduce a Dual‑Stream Salience‑Context Calibration Mechanism, which disentangles non‑verbal feature sequences into a focus stream and an ambient stream. The focus stream captures salient sentiment shifts (e.g., facial expressions) guided by textual priors, while the ambient stream characterizes stable background states. Through calibrating these dynamic sentiment shifts against background states, SentiLLM effectively projects non‑verbal modalities into a unified semantic space, making them naturally understandable for LLMs. Serving as a plug‑and‑play module, SentiLLM significantly improves discriminative performance with only a small number of trainable parameters. Our method achieves superior performance on four datasets, MOSI, MOSEI, CH‑SIMS, and CH‑SIMS v2, demonstrating the effectiveness of the structural abstraction paradigm in MSA. Our code is available at: \hrefhttps://github.com/especiallyW/SentiLLM.
Authors:Bowen Wang, Youwen Zhang, Ritesh Mehta
Abstract:
We describe the DS@GT submissions to the ImageCLEFmedical Caption 2026 challenge, which continues a long‑running benchmark on the ROCOv2 dataset with two tracks: Concept Detection (Task 1), assigning UMLS Concept Unique Identifiers (CUIs) to radiology images, and Caption Prediction (Task 2), generating natural‑language captions. For Task 1, our primary submission was a three‑way late‑fusion ensemble of ConvNeXt‑V2, BiomedCLIP ViT‑B/16, and DenseNet‑169 with a regularized ''Honest Threshold Tuning'' procedure designed to avoid validation overfitting on rare concepts; this submission ranked first on the official submission with a primary F_1 of 0.5790 and a secondary F_1 of 0.9657. In parallel, we submitted a training‑free KNN retrieval pipeline over frozen BiomedCLIP embeddings, which reached a primary F_1 of 0.5780 and a secondary F_1 of 0.9599‑essentially matching the fine‑tuned ensemble on the primary track at a fraction of the cost. For Task 2, our submissions included a fine‑tuned Gemma‑3 27B model (overall 0.3571, ranking third in the official submission), a fully fine‑tuned BLIP pipeline with custom Vizwins merging (0.3564), and a zero‑shot MedGemma‑4B run with a PubMed‑style prompt (0.3186), spanning a wide range of model scales and training costs. Code: https://github.com/dsgt‑arc/imageclef‑caption‑2026.
Authors:Peiyu Hu, Siying Gu, Weihai Lu, Zhuodong Liu, Yuntian Tang, Jiahao Liang, Yiying Xie, Jiang Rong, Zhaokai Luo, Zhiyong Wang, Jia Wang
Abstract:
Large Language Models (LLMs) have shown strong potential for recommendation by leveraging their semantic understanding and contextual modeling capabilities. Recent studies further introduce reasoning mechanisms to improve user preference modeling. However, explicit natural‑language reasoning incurs substantial inference overhead, whereas existing latent reasoning methods mainly focus on generating or verifying intermediate states, leaving their layer‑wise preference roles and contributions insufficiently characterized. We propose HiLaR, a Hierarchical Latent Reasoning framework with layer‑aware reinforcement optimization for LLM‑based recommendation. HiLaR constructs temporal‑guided hierarchical user preference representations, aligns them with multiple LLM latent reasoning states, and organizes the reasoning process from broad preferences to fine‑grained current intents. To further optimize the reasoning trajectory, HiLaR combines final recommendation feedback with layer‑aware process rewards derived from the marginal target‑likelihood gain of each state. Experiments on four Amazon benchmark datasets show that HiLaR generally outperforms strong sequential, generative, and LLM‑based recommendation baselines. Ablation and sensitivity analyses further verify the contribution of hierarchical representation learning, latent alignment, and process‑level optimization. Our code is available in https://github.com/hupeiyu21/HiLaR.
Authors:Sangmin Hong, Daniel Sungho Jung, Heewon Kim, Kyoung Mu Lee
Abstract:
Point clouds are one of the most fundamental and widely used 3D representations, serving as the most basic geometric representation of 3D shapes. Nevertheless, most existing 3D printing pipelines require a watertight mesh as input, preventing the direct use of point clouds for fabrication. A common workaround is to reconstruct meshes from point clouds; however, the resulting meshes often contain geometric artifacts, such as incorrect faces or topological inconsistencies, that are difficult to repair and may lead to printing failures. To overcome these limitations, we propose PrintAnything, a novel framework that learns to produce executable 3D printing G‑code directly from 3D point clouds without requiring mesh reconstruction. To enable point clouds to serve as direct input for slice‑wise toolpath generation, we introduce a slice‑wise point projection strategy that transforms unstructured 3D point clouds into slice‑aligned 2D representations consistent with layer‑by‑layer nature of fused deposition modeling in 3D printing. To eliminate mesh dependency and provide a unified representation that bridges point clouds and G‑code, we propose Geometric plan (G‑plan) map, a compact 2D representation composed of occupancy, region, and flow maps that encode the geometric and extrusion properties required for toolpath synthesis in 3D printing. As a result, our proposed method accurately generates printable G‑code directly from point clouds, enabling a practical and fully mesh‑free pipeline for 3D printing. The code is publicly available at \hrefhttps://github.com/Sangminhong/PrintAnythinghttps://github.com/Sangminhong/PrintAnything.
Authors:Shaobo Liu, Feiqiao Mao, Shuaishuai Zhou, Yan Zhan, Weiqi Tan, Zhiqiong Lu, Zhengping Liang
Abstract:
We propose RefineSVG, a single‑step closed‑loop visual feedback framework that enables multimodal large language models (MLLMs) to perform high‑fidelity image‑to‑SVG generation through self‑correction. Existing MLLM‑based approaches rely on single‑pass open‑loop inference, where the model receives visual input only once and must generate thousands of SVG code tokens without intermediate verification. This paradigm inevitably leads to geometric drift, error accumulation, and visual hallucination on complex images. RefineSVG overcomes this limitation by invoking an external rendering engine after an initial SVG generation pass to compare the rendered output against the target image. The comparison yields a multi‑dimensional visual residual map (Diff‑Map) that is fed back to the model as a ReAct‑style correction signal, driving a targeted correction step. To support this render‑observe‑correct interaction, we further introduce an SVG‑oriented semantic vocabulary that compresses token sequences by over 52%. A progressive training pipeline spanning supervised fine‑tuning, rejection‑sampling cold‑start data construction, and end‑to‑end agentic reinforcement learning aligns the model with closed‑loop visual correction. Extensive experiments show that RefineSVG consistently outperforms existing baselines in reconstruction fidelity, structural accuracy, and code efficiency.Code is available at https://github.com/liuxiaobo66/RefineSVG.
Authors:Jingya Wang, Yuyang Gao, Liuzhenghao Lv, Yonghong Tian, Yuyang Liu
Abstract:
We introduce LabEvolver, a training‑free framework that equips safe and grounded wet‑lab agents with episodic memory from execution experience. LabEvolver couples a state‑grounded inner trial loop for adaptive perception, online planning, and safety validation with an outer evolution loop that distills completed trajectories into reusable skill, strategy, and safety experience. On robotic solution‑preparation tasks, LabEvolver demonstrates real‑world feasibility, reducing pH‑regulation completion time and safety‑gate intercepts by 48.2% and 60.0%, respectively. On ALFWorld, it further improves cumulative success rate within 20 steps from 76.2% with ReAct to 91.4% over 500 continual tasks, showing generality beyond wet‑lab settings. These results support learn‑by‑doing experience evolution as a feasible path toward closed‑loop automated scientific discovery. The project page is available at https://andygao6186.github.io/LabEvolver/.
Authors:Fexiang Liu, Shiye Wang, Qiang Qiu, Zheng Wang
Abstract:
Reliable deployment of multimodal large language models (MLLMs) requires deciding whether a confident visual answer should be trusted, reviewed, or routed to a stronger system. Confidence scores capture candidate margins, but not where the estimated signed visual readouts associated with those margins come from or how they are distributed. We study inference‑time risk detection for closed visual answers using the same white‑box prefill path that produces the answer. Witness Evidence Portfolios (WEP) first estimates, layer by layer, which visual contributions support or contradict the predicted candidate. It summarizes these contributions through two interpretable route families: question‑related evidence provenance and signed evidence concentration. Nested grouped validation chooses the more reliable family and a sparse top‑k route portfolio, which is fused with candidate confidence. WEP needs no image perturbation, decoding change, backward pass, or external verifier. Across three MLLMs and four binary‑answer benchmarks, WEP improves mean error AP by 0.134. All 12 model‑‑dataset gains are positive, and image‑cluster bootstrap intervals are strictly positive on 10 pairs. WEP targets white‑box closed‑answer systems and uses a labeled calibration slice.
Authors:Yanning Hou, Haoyuan Chen, Sihang Zhou, Xiaoshu Chen, Xirui Liu, Duanyang Yuan, Lingyuan Meng, Siwei Wang, Quan Liu, Jian Huang
Abstract:
Reinforcement learning (RL) search agents commonly model retrieval as free‑form natural‑language query generation and optimize multi‑turn interactions using final‑answer rewards. Current studies mainly improve training with denser or more structured credit signals, but rarely examine whether retrieval is properly formulated at the policy‑environment interface. We observe pronounced retrieval aliasing during Search‑R1 training: rollouts for the same question continue to generate distinct query strings, yet their accumulated evidence sets increasingly overlap. We call this phenomenon retrieval‑equivalence collapse; in this regime, trajectories approach utility equivalence with respect to retrieval decisions, leaving within‑group returns with little effective retrieval contrast. To address this problem, we propose Harness‑G, a graph‑structured retrieval framework that redesigns this interface. It reformulates free‑form query generation as finite action selection: the policy selects an evidence sentence or entity, or chooses to answer, while the environment constructs the menu, tracks retrieval state, and validates and executes each choice. This interface reduces linguistic aliasing and makes same‑state alternatives directly comparable. Building on this interface, we introduce Structured Non‑myopic Credit (SNC), which uses a frozen answer scorer to compare the selected action with its alternatives and assigns downstream gains to the earlier actions that enabled them. Across six QA benchmarks, Harness‑G achieves the highest average F1 at both evaluated model scales, outperforming the strongest baseline, Graph‑R1, by 10.74 points at 1.5B and 3.98 points at 3B.
Authors:Wenjie Zhu, Yabin Zhang, Wenjun Zeng, Lei Zhang
Abstract:
Multimodal Large Language Models (MLLMs) have achieved strong performance on a wide range of vision‑language tasks, but often fail under imperfect or shifted contexts. A reliable MLLM should refuse truly out‑of‑context (OOC) questions with subject‑level context shifts while still answering shifted in‑context (Shifted IC) questions with non‑subject context shifts. Existing benchmarks mainly target OOC or visually unanswerable questions, but overlook answerable Shifted IC cases and cover limited OOC shifts. To fill this gap, we present MMOOC, a large‑scale benchmark for evaluating refusal and robust answering abilities of MLLMs. MMOOC contains over 41K image‑question pairs, including answerable Shifted IC cases and unanswerable OOC cases, spanning three question formats, eight shift types and six visual scenarios, with data quality ensured through MLLM‑based filtering and human verification. We evaluate model responses using Accuracy and Refusal Rate, and further introduce an LLM‑as‑a‑Judge metric to assess the correctness of model reasoning. Experiments on diverse MLLMs show that current models still struggle to balance answer‑ability and refusal under shifted contexts. We further analyze key failure patterns and show that post‑training can improve robustness. MMOOC will be made publicly available.
Authors:Zhenrong Zhang, Fei Wu, Jun Du, Jianshu Zhang, Si Wei
Abstract:
Reinforcement learning has emerged as an effective paradigm for enhancing the mathematical reasoning capabilities of large language models. Among existing policy optimization methods, Proximal Policy Optimization (PPO) remains particularly appealing because its learned critic can, in principle, provide token‑level credit assignment. However, in mathematical reasoning tasks characterized by long reasoning horizons and sparse outcome rewards, reliable token‑level credit assignment remains challenging. The standard critic often fails to accurately evaluate intermediate reasoning states, resulting in noisy advantage estimates and suboptimal policy updates. In this paper, we propose ReDiPPO, a Reference‑guided and Discrepancy‑aware PPO framework for mathematical reasoning. ReDiPPO introduces a reference‑guided critic that uses reference answers as training‑time privileged signals to provide more accurate value estimation. Meanwhile, it retains a standard critic and quantifies the token‑level reference‑standard discrepancy between the standard value estimate and the reference‑guided value estimate. This discrepancy serves as an indicator of difficult reasoning states and is used to reweight the corresponding token‑level advantages during PPO optimization. Extensive experiments on diverse mathematical reasoning benchmarks demonstrate that ReDiPPO improves value‑estimation accuracy and consistently outperforms strong policy optimization baselines, including PPO, DAPO, and GSPO, in final reasoning performance. Our code is available on https://github.com/cii030/ReDiPPO.
Authors:Stephen Gould, Anton van den Hengel
Abstract:
Key‑value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years. Computational demands of large language models (LLMs) and their multi‑modal variants during output generation can be partially alleviated by caching previous key and value calculations needed by subsequent scaled dot‑product attention operations. However, this leads to another problem: the size of the resulting KV cache grows linearly with context length and quickly consumes all available GPU memory when either the prompt or the generated output are long. KV cache management periodically prunes entries from the cache thereby reducing its memory footprint while attempting to retain sufficient information for accurate generation. A by‑product is faster inference speed. We propose a simple yet effective KV eviction scheme motivated by the insight that past tokens which can be well‑predicted from more recent tokens are redundant and their associated keys and values can be removed from the cache. To score entries for eviction we run the model on the tokens in their original order, reusing the key and value representations already stored in the KV cache, and applying a counter‑causal attention mask so that each position attends only to its future context. This is in‑distribution, tied directly to the actual cache contents, and requires no additional training. To further reduce cost, we additionally propose a fast single‑layer approximation that restricts the counter‑causal pass to the last transformer layer, achieving a significant speedup per refresh cycle at marginal accuracy cost. We evaluate our strategy on various open‑source LLMs and benchmark datasets showing competitive or improved performance over other state‑of‑the‑art methods. Reference code is available at https://github.com/metacognitionai/counter_causal.
Authors:Jinfan Zhou, Richard Liu, Itai Lang, Rana Hanocka
Abstract:
We present MeshFM, an efficient feedforward framework for extracting rich features from 3D inputs. Our method distills 2D features from visual foundation models into 3D. We train a feedforward network to directly predict 3D features without requiring optimization during inference. The approach utilizes a two‑stage training strategy. First, we optimize a feature field in 3D using only 2D feature supervision. Second, we train a network to regress this feature field. The entire procedure requires no 3D annotation, instead relying on the powerful information in 2D foundation models. We demonstrate that our learned features can be immediately applied to downstream tasks, including part segmentation, dense correspondence, and mesh deformation. Extensive experiments show that MeshFM, trained solely with 2D supervision, performs on par with methods trained explicitly with 3D supervision, even without task‑specific fine‑tuning. Moreover, our model is trained to be robust to extreme rotations of the input objects. Project page: https://threedle.github.io/MeshFM/
Authors:Mansoor Ahmed, Yue-Tsz Fan, Hemanth Venkateswara, Murray Patterson
Abstract:
Discrete diffusion and flow‑matching models denoise a sequence over many steps, but to keep each step cheap, they factorize the transition across positions and decide every token independently. This makes few‑step generation challenging for text when the target couples two positions, such as a subject and a verb that must agree. An independent update commits to them separately, and many function evaluations are spent repairing the mismatch. Existing few‑step methods buy back the lost correlation by distilling or rectifying a slow teacher, and so inherit the teacher's quality ceiling. We ask instead whether a model can express correlated steps natively, and answer with Latent‑Kernel Discrete Flow Maps (LKF), a from‑scratch flow‑map kernel that is a mixture of M factorized components tied by a single shared latent. Conditioned on the latent, each component is cheap, and the mixture is summed over the latent in closed form for small M. We show that a single step places mass on correlated completions with the same sampling time complexity as a factorized model, since one latent is drawn per sequence and reused across the entire denoising trajectory. We also show that the Masked Diffusion Language Model (MDLM) is a special case of our LKF model at M=1. The experiments for unconditional text generation on the One‑Billion‑Word (LM1B) and WikiText‑103 benchmarks show that our LKF model learns strongly heterogeneous components and improves generative perplexity by 2.1x to 3.3x over the likelihood baselines without losing diversity. The gain grows with M, and at M=8, it surpasses distilled and rectified few‑step samplers. The source code is available at: https://github.com/mansoor181/lkf.git
Authors:Gabriel K. Gegenhuber, Moritz Grefner, Maximilian Günther, Matthäus Wininger, David Schmidt, Aljosha Judmayer
Abstract:
End‑to‑end encrypted (E2EE) messaging apps are widely praised for their security and thus also used for sensitive coordination in group chats (e.g., by political decision makers). After Threema and WhatsApp, also Signal and iMessage have recently introduced polls to aid agreement processes in groups. This implicitly sets the expectation that all participants see the same outcome and thus have the same view of the conversation. This property is commonly referred to as transcript consistency (TC). In this work, we demonstrate that today's major E2EE messengers do not guarantee any form of TC for group chats, allowing a malicious group member to selectively omit, reorder, or present altered content to different recipients without triggering warnings in their user interface. We systematically investigate the extent of the problem under a malicious‑participant threat model that targets the integrity of the shared transcript, or inconsistent delivery across a user's linked devices. We identify multiple equivocation vectors that range from protocol fallback paths to deliberate use of pairwise delivery channels within groups. We demonstrate concrete exploitation scenarios such as social engineering, evading moderation, and, in particular, rigging polls. Beyond these cross‑service design issues, we also uncover implementation‑specific behaviors with privacy implications (e.g., device OS fingerprinting). Finally, we contextualize our findings within prior transcript‑consistency research and outline practical low‑overhead mitigations and UI signaling strategies that can be integrated into state‑of‑the‑art E2EE group protocols.
Authors:Joshua Meyer, Sahar Shayegan, Ritiz Tambi, Ali Khan, Sun Kim, Victor Shih, Mehdi Jamei, Andi Partovi
Abstract:
Production voice agents span cascaded, speech‑to‑speech, and hybrid architectures. Voice‑agent benchmarks typically measure component quality and conversational properties such as word error rate, latency, naturalness, and turn‑taking. Fewer measure whether the agent handled a phone call correctly on its own. Contact centers refer to this as ``containment'': the share of phone calls the automated system resolves without handing off to a human. On some phone calls the right outcome is refusal or a redirect. To address this gap, we introduce VAmoS Bench, the Voice Agent Simulation Bench. It measures complete voice‑agent systems end to end in a stateful customer‑support task. The agent is Riley, a credit‑card support representative for a fictional bank who can freeze, cancel, replace, or activate a card. Each of 100 scenarios supplies a simulated caller with a private goal and a seeded PostgreSQL backend. The platform uses each scenario to populate and activate an isolated simulation in which the caller reaches Riley over audio; roughly one‑third apply adversarial pressure. The agent can use five tools that execute real SQL against the backend. Each scenario also defines binary assertions. A grader evaluates them against the complete trace of what the caller and agent said and what the agent did, including tool invocations, arguments, and returned rows. This catches an agent that claims to have changed a card without updating the database, as well as one that makes the right database change while disclosing protected information. This first benchmark version focuses on financial services. Its evaluation protocol supports an evolving leaderboard: additional voice agents can be evaluated on the same version, while later versions can expand the tasks and scenarios.
Authors:Ru Peng, Tianyu Zhao, Xijun Gu, Zhiting Fan, Haokai Xu, Jinyang Zhang, Yawen Zeng, Yihong Zhuang, Kexin Yang, Junyang Lin, Dayiheng Liu, Junbo Zhao
Abstract:
High‑quality, diverse data are vital for large language models (LLMs) but remain scarce and costly. Data synthesis is a viable alternative and succeeds on closed tasks, yet the humanities and social sciences (HSS) are overlooked, and their open‑ended nature makes synthesis challenging. Moving beyond prior capability‑centric, fragmented attempts, we adopt a subject‑centric paradigm, define the first HSS domain system covering 14 mainstream fields, and introduce HSS‑Synth, the first data synthesis pipeline for HSS. HSS‑Synth comprises: (1) constructing seed documents from web corpora via multi‑step filtering and text refinement evaluated by a judge; (2) specifying "requirements + persona" to backtranslate seed documents into diverse yet faithful instructions with a strict Q&A alignment check; and (3) breaking LLM response limits via teacher‑forced Answering that feeds seed documents during response generation to anchor semantics, reduce hallucinations, and preserve tone and integrity. HSS‑Synth yields 237k high‑quality, diverse instruction‑tuning samples that outperform 14 leading baselines on 16 benchmarks. The fine‑tuned Qwen3‑8B‑Base sets a new SOTA and approaches the official Qwen3‑8B, improving both human preference and knowledge capabilities without performance seesaws. Extensive experiments demonstrate HSS‑Synth's robustness and transferability. Our code is publicly available at https://github.com/pengr/HSS‑Synth.
Authors:Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le, Hyunseung Choo
Abstract:
Cross‑modal knowledge distillation can transfer diagnostic knowledge from a strong but costly teacher modality to a cheaper and more deployable student modality. In medical image analysis, however, the two modalities are often unpaired: they are collected from different patient cohorts and occupy geometrically incompatible feature spaces. This makes instance‑level distillation invalid and direct feature matching unreliable. To address these challenges, we propose Shared Semantic Codebook Distillation (SSCD), which compares teacher and student representations through a shared discrete codebook. Each image is represented as a distribution over a common, modality‑agnostic vocabulary, and knowledge is transferred by aligning these distributions across modalities, both globally and class‑conditionally, without requiring paired samples or directly comparable raw features. The codebook is evolved online by exponential moving average and kept diverse through entropy regularization and dead‑code restart. At inference, all teacher‑side and codebook modules are discarded, leaving only the student encoder and classifier. On two heterogeneous unpaired settings, OCT‑to‑fundus retinal disease classification and CT‑to‑chest‑X‑ray pneumonia classification, SSCD improves the student from 64.5 to 70.2 macro‑F1 and from 73.8 to 76.3 macro‑F1, respectively, outperforming all evaluated distillation baselines on both settings. Code and pretrained models are available at https://github.com/DillanImans/SSCD‑unpaired‑distillation
Authors:Musa Shams
Abstract:
Agentic retrieval‑augmented generation systems can produce answers that appear grounded while failing at the evidence, tool‑contract, authorization, or session‑state layer. We introduce LayerRAG‑Bench, a controlled cross‑layer reliability benchmark with 8 enterprise domains, 240 tasks, 9 fault scenarios, 2 contract modes, and 38,880 live task‑level records across nine models from OpenAI, Anthropic, and Gemini. Schema normalization raises schema‑drift success from 0.000 to 0.913, but stale evidence, missing tool output, denied permissions, and wrong‑session context are not recovered by schema normalization. Groundedness‑only evaluation also produces substantial false positives under stale and wrong‑session evidence. These results support a layer‑specific evaluation principle: a reliability intervention should be credited for repairing its target layer without being mistaken for a universal fix.
Authors:Joonhyung Bae, Dawon Park, Taegyun Kwon, Yoon-Seok Choi, Hyeon Hur, Satoshi Obata, Shigeru Kai, Yohei Wada, Yu Takahashi, Akira Maezawa, Jaebum Park, Jonghwa Park, Juhan Nam
Abstract:
Music information retrieval research on piano performance increasingly involves diverse modalities of data and annotations beyond audio and MIDI. We present SKY‑Piano, a multimodal piano performance dataset that includes 11 hours of performance recordings of motion, multi‑view video, audio, MIDI from 7 professional and 12 amateur pianists along with MusicXML scores. The performance pieces were selected considering playing technique, difficulty, and performer expertise on a shared core repertoire. The motion data include both hand and body motion, released in both flagged form, where samples lost to marker occlusion are marked as unreliable, and imputed form, where those gaps are reconstructed, together with Visual3D body‑segment kinematics and other time‑synchronized modalities. To easily browse different modalities of data at a glance, we provide an interactive web browser. In addition, we developed a fingering annotation model and tool for deriving pseudo fingering annotations from the MIDI and motion data. Lastly, we present MIDI‑to‑motion generation through a fine‑tuning experiment as a use case of the dataset.
Authors:Gal Engelberg, Michael Arenzon, Leon Goldberg
Abstract:
Enterprises are moving toward autonomous cyber defense: agentic AI that builds situational awareness of an organization's security state and reasons from it to assessments, decisions, and actions. This rests on a holistic view of the enterprise's security state, the continuous, cross‑vendor picture of identities, cloud and infrastructure, data, applications, and their configurations that security posture management assembles. As agents take on this work, what matters is not whether an agent can produce an answer but whether it should be trusted to. The field cannot yet answer this question. Real enterprise environments are private, cross‑vendor, and deeply correlated, and none is exposed publicly as a shared, queryable target for evaluating such agents end to end. We call this the environment data gap. We present Open Security Benchmark (OSB), a framework that benchmarks agentic AI on this work. OSB surfaces a curated enterprise environment ‑ a frozen, holistic view of the security state ‑ and evaluates posture investigation across two modalities: text‑to‑SQL over a relational snapshot and each vendor's native API over a served instance of the same environment. Freezing the environment pins the target state as an immutable snapshot and anchors answers to a closed‑form ground truth. OSB is built from five components: a data layer, a task and evaluation‑set layer, a multi‑dimensional scoring layer, a minimal auditable harness, and a bring‑your‑own path that serves public comparison and private tenant evaluation from one substrate. We instantiate the framework with two identity‑security packs and a family of synthetic‑organization environment datasets spanning multiple scales, and chart its extension to further posture subdomains, investigation modalities, and defense stages from assessment toward remediation.
Authors:Kaiyu Li, Zepeng Xin, Zixuan Jiang, Jing Fu, Lanxuan Xue, Lingyu Zhang, Xiangyong Cao
Abstract:
Open‑vocabulary Earth observation (EO) aims to localize geospatial concepts specified in natural language rather than a fixed label set. Existing benchmarks, however, usually cover narrow category vocabularies or limited query forms. To fill this gap, we introduce OVEarth‑Bench, which extends existing evaluation in two directions: category breadth, through broad hierarchical category coverage with positive and negative expressions, and query diversity, through vocabulary, referring, and reasoning queries. The benchmark supports mask and box localization under a unified zero‑shot protocol. We evaluate a broad set of general and EO‑specific methods. The evaluation reveals that: (1) the performance of current methods remains limited, while broader category coverage yields more stable model rankings; (2) MLLM‑based methods achieve the strongest overall performance; and (3) EO‑specific methods generally underperform general models and rarely match the strongest methods. These findings provide guidance for future open‑vocabulary EO method design and highlight the importance of developing more realistic, diverse, high‑quality, and large‑scale benchmarks for reliable evaluation. Our data and evaluation package are released at https://earth‑insights.github.io/OVEarth‑bench.
Authors:Weiye Shi, Fanxu Meng, Muhan Zhang
Abstract:
Multi‑head latent attention (MLA) is increasingly important for long‑context LLM inference because compact latent states replace the growing key‑value (KV) cache and reduce decoding memory traffic. Yet most capable open checkpoints use multi‑head or grouped‑query attention (MHA/GQA), so conversion is needed to obtain MLA's cache efficiency without retraining from scratch. Speculative decoding offers complementary acceleration, but its speedup depends on agreement between draft proposals and target verification. We find that direct MHA/GQA‑to‑MLA conversion can sharply reduce this agreement: low‑rank factorization and RoPE handling introduce attention‑function errors that may be tolerable for standalone generation but substantially lower draft‑token acceptance. We therefore formulate MLA draft construction as functional reconstruction rather than cache compression. Our end‑to‑end (E2E) method optimizes each converted MLA attention module to reproduce the post‑output‑projection response of its original MHA/GQA counterpart on calibration hidden states. This converter‑agnostic post‑conversion procedure preserves the converted cache and inference graph and requires neither verifier logits nor verifier supervision. We evaluate 192 model‑converter‑backend‑method‑task configurations spanning four Llama/Qwen draft‑target pairs, TransMLA and MHA2MLA, HF and vLLM, and four 200‑prompt tasks. With a 0.5‑percentage‑point reporting tolerance, Functional Reconstruction materially improves acceptance in 37 of 64 matched task cells, leaves 26 practically unchanged, and materially decreases one. Code and evaluation artifacts are available at https://github.com/swyhahaha/FunctionalMLA.
Authors:Frank Li, Bardia Khosravi, Mohammadreza Chavoshi, Theo Dapamede, YoungSeok Jeon, Janice Newsome, Hari Trivedi, Judy Gichoya
Abstract:
Training deep learning models on radiological images requires integrating heterogeneous datasets across different sources, file formats, directory layouts, label schemas, and annotation types. We present RadHarmony, an open‑source Python library that provides a unified API for loading, harmonizing, and augmenting radiological datasets, with a primary focus on chest radiographs and early support for computed tomography (CT) and magnetic resonance imaging (MRI). RadHarmony standardizes metadata from 24 public datasets into a single tabular format, wraps MONAI's map‑style datasets for deep‑learning‑ready sample delivery with optional on‑disk caching, and supports classification labels, segmentation masks, bounding boxes, and radiology report text through a single interface, with an interactive visualization tool for dataset exploration and verification. To lower the barrier for integrating new datasets, RadHarmony introduces an AI‑agent skill that guides the full integration workflow from raw data inspection through code generation and testing. We demonstrate the library's utility by pretraining RadHarmony‑ViT, a reference vision transformer baseline that combines three heterogeneous chest radiograph datasets with no dataset‑specific code. The code and pretrained model weights are available at https://github.com/f10409/RadHarmony.
Authors:Peiyu Zang, Jian Tao, Jialing Zhang, Yichen Yuan, Wentao Zhang, Guang Liu, Yonghua Lin
Abstract:
Large language models (LLMs) have significantly increased the demand for efficient accelerator kernels, but kernel development remains a highly specialized and labor‑intensive task. The recent rise of LLMs and agentic frameworks offers a promising pathway toward automatic kernel generation. However, despite rapid progress, there is still no comprehensive benchmark to rigorously evaluate LLM‑generated kernels across diverse operator sources or heterogeneous hardware platforms. We present KernelGenBench, a unified benchmark for systematically evaluating LLM‑ and agent‑generated Triton kernels across diverse operator sources and heterogeneous hardware platforms. It comprises two complementary sub‑benchmarks: KernelGenBench‑MS (Multi‑Source), evaluating 210 operators from three sources beyond standard PyTorch‑centric tasks, and KernelGenBench‑MC (Multi‑Chip), measuring performance portability across six heterogeneous hardware platforms using a 110‑operator subset. Our large‑scale evaluation, consuming over 15 billion tokens, shows: (1) agent‑based methods consistently outperform pure LLM sampling methods, while cuBLAS operators are the most challenging across all methods; (2) generation performance varies significantly across hardware platforms, with even recent kernel‑specialized agents experiencing severe cross‑platform degradation (e.g., AutoKernel drops from 87% on NVIDIA to 25% on Platform E); (3) autonomous kernel generation remains highly cost‑intensive, with specialized agent methods averaging 5.11 million tokens per successful operator (AKO4all reaches 5.19 million), orders of magnitude higher than simple LLM sampling approaches.
Authors:Hengyi Xie, Chenfei Yao, Xianjin Wu, Xuanyang Xi, Yiping Tang, Di Xu, Yingying Zhu, Dingkang Liang, Xiang Bai, Han Ding
Abstract:
Vision‑language‑action (VLA) models commonly adopt an LLM‑centric V \to L \to A pathway, where visual observations are projected into the representation space of a large language model before being decoded into robot actions. Although effective, this design incurs substantial computation and memory overhead at every policy invocation. In this work, we introduce TurboVLA, a new VLA paradigm that reformulates the conventional V \to L \to A pathway as a direct V + L \to A mapping. Instead of using a large language model as the central interface between perception and action, TurboVLA independently encodes visual observations and language instructions, directly exchanges information between them through lightweight bidirectional vision‑language interaction, and predicts continuous action chunks with a compact decoder. This simple design constructs task‑conditioned representations directly from visual and linguistic features, significantly reducing the computational and memory costs of VLA inference. On LIBERO, TurboVLA achieves 97.7% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GB inference VRAM on a consumer‑grade RTX 4090, matching or outperforming substantially larger VLA policies. These results establish TurboVLA as a simple and effective alternative to the prevailing LLM‑centric VLA paradigm, offering a new perspective on how vision, language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H‑EmbodVis/TurboVLA.
Authors:Zador Pataki, Paul-Edouard Sarlin, Marc Pollefeys
Abstract:
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large‑scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over‑optimized for real‑time operation and generally requires known camera calibration; while Structure‑from‑Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide‑baseline dense image matching, treats temporal ordering as a first‑class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state‑of‑the‑art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.
Authors:Hao Tan, Jun Lan, Zichang Tan, Ajian Liu, Zijian Yu, Chuanbiao Song, Huijia Zhu, Weiqiang Wang, Jun Wan, Zhen Lei
Abstract:
The growing capability of image generation models has made synthetic images a routine presence in open media, making robust and generalizable AI‑Generated Image (AIGI) detection increasingly essential. While multi‑modal large language models (MLLMs) offer a transparent alternative to black‑box binary scoring, we observe that current MLLM‑based detectors still exhibit notable perception bottlenecks in capturing fine‑grained anomalies. They primarily focus on how visual evidence is organized and synthesized, leaving the intrinsic perception less optimized. To mitigate this gap, we present Veritas++, a perception‑enhanced reasoning framework that establishes reliable perception as the foundation of authenticity reasoning. Rather than directly optimizing the model's explanatory ability, we ground AIGI detection on three basic perception abilities, i.e., capturing fine‑grained visual details, semantic anomalies and pixel‑level differences. Building on this insight, we introduce Perception‑oriented Learning (PoRL), which replaces open‑ended description supervision with verifiable rewards to explicitly strengthen these capacities. To further integrate enhanced perception with reasoning, we introduce Value‑aware On‑Policy Distillation (VaOPD), an adaptive distillation mechanism that prioritizes high‑value distillation signals over uniform supervision, internalizing perception‑aware reasoning through a privileged self‑teacher. Extensive experiments across standard, in‑the‑wild and emerging benchmarks demonstrate that Veritas++ achieves promising generalization. The perception learning effectively bridges the perception gap and yields seamless gains on detection, while VaOPD further enables efficient capability evolvement without sacrificing existing performance. Code and checkpoints are available at https://github.com/EricTan7/VeritasPP.
Authors:Filipa Lino, Bárbara Tavares, Carlos Santiago, Cláudia Soares, Manuel Marques
Abstract:
Emergency Departments (EDs) are critical access points in healthcare systems, yet they face persistent pressure from unpredictable patient demand, seasonal surges, and non‑urgent visits. Effective ED planning requires forecasts at multiple decision‑making levels: hospitals need local demand estimates for staffing and bed management, regions require forecasts to coordinate healthcare units, and national authorities need system‑wide projections for capacity planning. However, most existing approaches forecast ED demand independently at a single level, ignoring the hierarchy linking hospitals, regions, and national systems. This can produce incoherent predictions, where hospital‑level forecasts do not aggregate consistently to regional or national demand. We propose HierSTT, a hierarchical Transformer‑based framework for coherent multi‑level ED forecasting. HierSTT jointly predicts hospital, regional, and national level demand in a single end‑to‑end model. A Temporal Fusion Transformer captures national dynamics, while spatio‑temporal Transformer encoder‑decoder modules model regional and hospital demand conditioned on higher‑level forecasts. A coherence‑aware loss penalizes cross‑level inconsistencies during training. We further introduce a nationwide Portuguese ED dataset covering 81 hospitals across 5 regional health administrations, with heterogeneous covariates at each level. Experiments show that HierSTT reduces average WAPE by 32% relative to the best non‑hierarchical deep learning baseline and outperforms all classical hierarchical reconciliation methods, while producing near‑coherent predictions across levels. Additional resources associated with this work are available at https://github.com/FilipaLino/HierSTT.
Authors:Aleksandr Berdnikov, Yevgeny Liokumovich
Abstract:
We analyze whether language models of size ~100B have a representation of the night sky map that is decodable from their residual stream. We find that most of the considered open‑source models do have such a representation, and it often even surfaces to the top principal components on prompts that ask questions like ``what is close to this object in the night sky''. In all but one model this representation showed significant scores in LOO testing, containing up to 65‑85% of variance (R^2‑score) and having median angular error down to 12^\circ‑21^\circ. We verify that our representation is not a simple leak from a correlated flat representation. To our knowledge, this representation is the first example of a curved high‑dimensional irreducible feature manifold. Codes used in the paper are published at https://github.com/l3erdnik/Decodable‑sky
Authors:Zihan Deng, Chuanzhi Xu, Huiqi Liang, Haoyang Li, Xiaozhen Zhong, Lequan Yu
Abstract:
Scientific images are the core elements of presenting experimental conclusions, elaborating system architecture, and supporting comparative arguments in scientific papers. However, existing image quality assessment (IQA) methods are predominantly designed for natural photographs or AI‑generated content, which cannot be directly applied to scientific papers. The few existing studies on scholarly charts remain confined to visual‑surface comparisons, failing to verify caption alignment, citation relevance, or visual misleadingness. To address this, we propose SciFigQual‑Bench, a full‑text contextual benchmark that evaluates scientific images across five dimensions (clarity, layout, caption fit, context relevance, and misleading risk). The data covers top computer‑science conferences from 2020 to 2025; 6,308 images were independently scored by multiple domain experts in five dimensions and aggregated into gold‑standard annotations. Unlike previous scientific figure benchmarks, our dataset binds each image to its caption, citing sentence, and manuscript context. To enable automated evaluation on this benchmark, we designed a staged cross‑modal evaluation framework SFQ‑Agent to achieve auditable and refined scoring through the collection and fusion of modal evidence. Multiple mainstream large models were evaluated on the test subset eval1200, and SFQ‑Agent (F3) equipped with GPT‑5.6‑Sol achieved the lowest overall average absolute error (0.418) and the highest consistency rate (93.4%), consistently outperforming both direct evaluation and auxiliary (Sidecar) visual language model evaluation schemes.
Authors:Kindeep K. Dhatt, Tengyue Wu, Hanbang Hua, Yayun Du
Abstract:
Continuous cuffless blood pressure (BP) monitoring remains challenging due to motion artifacts, physiological variability, and the limited robustness of conventional pulse transit time (PTT) models under dynamic conditions. Many prior approaches rely on multi‑second windows to stabilize estimation, an assumption that is frequently violated during real‑world monitoring with intermittent signal corruption. Here, we show that discriminative BP‑related information is preserved at the single‑beat level and present a lightweight multi‑modal wearable framework for continuous BP estimation. The system integrates synchronized chest electrocardiography (ECG) and ear‑clip reflectance photoplethysmography, each co‑located with a 6‑axis inertial measurement unit to provide motion context. We introduce a hybrid learning architecture in which a one‑dimensional convolutional neural network extracts a 64‑dimensional embedding from individual PPG beats and fuses it with 30 physiology‑grounded features, including PTT statistics and heart rate variability, followed by LightGBM regression. The method was evaluated using a multi‑phase stress protocol (n=10) and the PulseDB public dataset with subject‑disjoint validation. Across 30 independent runs, the model achieved mean absolute errors of 4.02 \pm 0.21~mmHg for systolic BP and 1.79 \pm 0.05~mmHg for diastolic BP, corresponding to a 28.2% reduction in combined MAE relative to baseline models. By enabling beat‑wise estimation without long temporal context, this framework supports computationally efficient cuffless BP monitoring suitable for wearable deployment under practical resource constraints. The source code for this work is available at https://github.com/SYMBIOX‑Lab/BP‑wireless.
Authors:Feixiang Liu, Qiang Qiu, Lanbo Sun, Nan Wei, Huawei Shen, Xueqi Cheng
Abstract:
Closed yes/no spatial benchmarks can reward a correct answer even when the image adds little support beyond no‑image contexts. Under a fixed forced‑choice interface, Visual Credit Audit (VCA) separates two estimands: whether the benchmark image gives the model's declared decision more support than text‑only and blank controls, and whether the model responds to relation‑specific visual evidence. The first audit is training‑ and label‑free and does not require an answer flip. Applying labels yields dependence‑credited correctness (D‑CC); on correct items, it equals same‑control gold‑aligned positive gain, while prediction alignment extends the audit to errors. Across four open MLLMs and two spatial benchmarks, 12.73‑26.25% of decisions are correct yet uncredited. Matched same‑split image permutation reduces D‑CC by 21.25‑47.80 points, with every paired 95% interval above zero. Fixed‑pixel relation contrasts and a 3x3 evidence‑source factorial show why null controls cannot identify relation response. Among controlled correct‑but‑uncredited agreement decisions, response to relation reversal spans 81.57‑100.00%, while 32.11% pooled change answer. Independently audited outcomes on 108 geometry‑compatible edits provide a bounded natural‑image correspondence check. VCA thereby decomposes benchmark success into correctness, additional image support, and relation‑consistent response.
Authors:Hendrik Ranocha
Abstract:
Although entropy‑based summation‑by‑parts (SBP) discretizations of hyperbolic conservation laws are widely used for their robustness and stability properties, there are very few results on their convergence. We extend a recent convergence analysis of Worku, Del Rey Fernández, and Zingg (2026, DOI: 10.48550/arXiv.2603.18369) in two ways. First, instead of allowing only hyperbolic conservation laws whose fluxes are homogeneous and have globally bounded second derivatives (a restriction essentially to linear or quadratic fluxes), we consider general hyperbolic systems with strictly convex entropy and source terms depending on time and space. Second, instead of requiring a special class of SBP operators, we consider a general framework of diagonal‑norm SBP operators on curved meshes, including finite differences, continuous and discontinuous Galerkin methods. Since the error analysis is based on a discrete relative entropy, it is restricted to smooth solutions. To enable a unified treatment of conservation laws, we restrict the analysis to periodic boundary conditions. Numerical results demonstrate that the predicted convergence rates are sharp in general, but can be improved for special cases such as discontinuous Galerkin methods with even polynomial degree and multi‑block finite difference methods. An optimal analysis is expected to require more sophisticated arguments specialized to the class of methods instead of the general framework of SBP operators used in this work.
Authors:Petr Simecek, Elnaz Babayeva, Jiri Balhar, Michal Bida, Michal Buran, Vaclav Cadek, Luigino Camastra, Tomas Dulka, Michal Janocko, Tomas Klohna, Pavel Kohout, Ondrej Kokes, Adam Krivka, Jakub Kubik, Patrik Mada, Igor Morgenstern, Marek Pavelka, Joshua Rogers, Petr Stastny, Jan Tattermusch, Dmitrijs Trizna, Martin Votruba, Guido Vranken, Jakub Zikl, Evelina Gabasova, Stanislav Fort
Abstract:
LLM‑based analyzers have begun finding real vulnerabilities in mature open‑source projects: AISLE's analyzer is credited with more than 280 CVEs across 78 projects, including OpenSSL, curl, and GnuTLS. We introduce HoF‑Bench (named after AISLE's public Hall of Fame), a benchmark built from 95 of these public AI‑discovered CVEs across eight repositories pinned at vulnerable commits. Analyzers receive source and target‑file scope but not CVE identifiers, descriptions, fixes, or expected mechanisms; a detector‑blinded frontier‑model judge credits only findings that identify the same code path, root cause, attack condition, and impact. A deliberately minimal LLM‑based analyzer rediscovers up to 65 of the 95 CVEs (68%) under this strict protocol. No frontier model performs detection anywhere in the study. The ten detector backbones are five open‑weight models (21B‑‑284B total parameters, 3‑‑13B active) and five proprietary small or "flash"‑tier models. All of them run in the fixed scaffold with four repeated passes, an optional generated‑context stage, and a replayable multi‑round triage stage (7,600 model‑‑CVE pass records). Difficulty is strongly structured by language; the CVEs missed by every model concentrate in C infrastructure code. HoF‑Bench provides a compact test bed for comparing vulnerability scanners, their reliability across repeated runs, and the candidate volume they create. The dataset is available at https://huggingface.co/datasets/aisleinc/HoF‑Bench.
Authors:Jinhu Qi, Wentao Zhang, Siu Man Ng, Feiyang Xu, Yanyu Chen, Yaoman Li, Irwin King
Abstract:
Travel planning is a demanding stress test for tool‑using LLM agents: a usable itinerary is a single artifact that must be right along many axes at once ‑ every flight, hotel, and attraction must exist and be bookable, the days must be physically traversable, the total must clear a budget, and the plan must serve a traveler whose needs are only partly stated. Existing agent benchmarks reward these properties one at a time and grade the final output with soft or LLM‑judged rubrics, which cannot certify that a returned plan is executable and are neither reproducible nor auditable. We introduce TREK (Travel Reasoning and Evaluation Kit), a benchmark for feasible itinerary synthesis: producing a single plan that is jointly constraint‑correct, hallucination‑free, spatio‑temporally executable, budget‑valid, and responsive to the traveler's unstated persona needs. TREK comprises 800 multi‑constraint tasks ‑ 533 feasible and 267 provably infeasible with typed route/entity/budget causes ‑ over a synthetic, internally consistent knowledge base of 212,530 records across 375 cities and 13 personas, served through a production‑style tool sandbox of validated RESTful APIs. Every task is scored by a fully deterministic, rule‑based evaluator with no LLM judge and ships a human‑verified gold reference that scores a perfect 1.0 under that same evaluator, so the ceiling is demonstrably achievable and every remaining gap is an agent limitation rather than scorer strictness. Evaluating 15 LLM agents across nine constraint dimensions, we find that even the strongest (GPT‑5.6) produces a fully‑feasible plan on only 46.2% of solvable tasks, with a median of 6.6% and a floor of 0.0%; satisfying travelers' unstated needs emerges as the universal bottleneck, unsolved even at the frontier. We release the dataset, tool sandbox, deterministic evaluator, and agent code as a fully reproducible benchmark.
Authors:Weile Gong, Zijian Lu, Mingcai Chen, Yiping Zuo, Xin He, Weibei Fan
Abstract:
Vision‑language models often use descriptions of earlier visual states to make decisions about the current scene. When the scene changes, stale language can redirect an otherwise correct visual judgment toward an outdated answer. We study this failure as visual lock‑in in a controlled grounding setting where only the verbalized prior varies. Across models, stronger lock‑in accompanies smaller changes in the model representation before the final answer. This reversal suggests that lock‑in depends not on how far this representation moves, but on how that movement is organized. In models that are harder to correct, prior‑induced changes concentrate along a compact set of directions that repeatedly appear across examples. We call these recurrent axes the Prior Directions. They recur on held‑out examples, while a descriptive four‑model comparison associates greater concentration with stronger lock‑in. Controlled interventions show that removing the component aligned with the Prior Directions restores visual grounding, whereas removing an equally large orthogonal component has little effect. Prior control thus arises when prior‑induced changes form a coherent and reusable pattern in the representation used to produce the answer. This account explains why the same prior remains revisable in one model yet becomes dominant in another.
Authors:Jianze Wang, Kunwang Zheng, Ying Liu, Yu Cao, Qilong Zhang, Jinlong Chen, Hua Yang, Qianglong Chen
Abstract:
Test‑time reinforcement learning (TTRL) enables language models to self‑evolve at inference time without labeled feedback. Existing methods rely on answer voting and therefore do not extend naturally to open‑ended generation, where valid responses cannot be mapped to a shared canonical answer. Without external reward models or stronger judges, adaptation must instead construct reliable rewards from the model's own outputs. We introduce SERPO (Self‑Evolving Rubric Policy Optimization), which replaces answer voting with a closed loop that co‑evolves response evidence, query‑specific rubrics, and policy parameters. Good‑Normal‑Bad (G‑N‑B) response evolution organizes maximally separated rollouts into ordered archives; rubric evolution retains criteria that discriminate these archives; probabilistic criterion scoring converts verdict‑token likelihoods into reward signals; and policy evolution optimizes the actor with the resulting signals. New actor rollouts then refresh both the archives and rubrics, closing the three‑way evolution loop. Across two model configurations, two in‑domain benchmarks, and four OOD benchmarks, SERPO improves HealthBench and ResearchQA by up to 20.63 and 20.31 points over the corresponding base models, raises the six‑benchmark macro‑average by up to 8.06 points, and supports OOD transfer and continued cross‑benchmark evolution.
Authors:Wenze Liu, Xintao Wang, Pengfei Wan, Xiangyu Yue
Abstract:
We propose amortized moment matching, utilizing neural networks to learn data moments as distributional training signals. By casting diffusion denoisers through polynomial projections, we establish a general framework for moment amortization, revealing that an n‑th degree projection explicitly identifies data moments up to order n+1. Derived from the tractable affine case, we instantiate the Amortized Fréchet Distance (AMFD) loss. Unlike FD‑loss which relies on explicit marginal moment calculations, AMFD is able to dynamically learn conditional moments via an alternating, matrix‑free optimization pipeline that effortlessly scales to high‑dimensional data. When operating on global representation features, AMFD serves as a powerful post‑training objective; empirically, its neural formulation yields more robust training dynamics than exact statistical matching, substantially surpassing the FD baseline on the FDr^6 metric and achieving superior one‑step generation on ImageNet. Furthermore, it unlocks direct exploration within native generative spaces, suggesting that the first two moments can identify target distributions only in spaces with strong semantics. Finally, when scaled to text‑to‑image generation, the condition‑aware nature of AMFD unlocks massive gains in instruction‑following capabilities, enabling our one‑step models to outperform their multi‑step FLUX.2 [klein] 4B teachers on the GenEval benchmark while achieving on‑par performance on PickScore. Code and checkpoints are available at https://github.com/poppuppy/amfd.
Authors:Jiaxing Li, Kai Zou, Cindy Zhou, Kaichen Huang, Junyao Gao, Zile Wang, Yang Liu, Bin Liu, Bo An, Yangguang Li
Abstract:
Existing autoregressive video distillation methods commonly adopt a Distribution Matching Distillation (DMD)‑based multi‑stage pipeline. However, they typically decouple the initialization and DMD stages ‑‑ which then pursue different target distributions ‑‑ and judge the intermediate student mainly by visual scores such as VBench. In this paper, we revisit this design from a distributional perspective. Given the mode‑seeking nature of the distribution matching loss, a good initialization should match the mode coverage of the target DMD teacher, rather than merely pursuing high quality. To analyze this, we introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. It exposes differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode‑covering ones preserve broader support. Furthermore, even when the target distributions are aligned, DMD's reverse‑KL objective can still drive the student toward high‑probability teacher regions in late training, reducing coverage and diversity. To address this, we propose joint distillation, which combines DMD's mode‑seeking objective with a Consistency Distillation‑based mode‑covering constraint. Experiments show that our method improves generation quality, coverage, and diversity; notably, even with a Wan‑1.3B DMD teacher, it outperforms baselines refined with Wan‑14B, underscoring the importance of distributional alignment in autoregressive video distillation.
Authors:Zhaoyang Ma, Zhihao Wu, Xin Gao, Lipo Wang, Youfang Lin, Jing Wang
Abstract:
Federated learning (FL) enables collaborative learning over decentralized data silos without centralizing raw data. However, heterogeneous local architectures often induce non‑aligned representation spaces, making it difficult to transfer global knowledge across silos. Existing paradigms share this knowledge as model parameters, distilled predictions, or class prototypes, yet all encode it in an absolute space that must be aligned across clients. Heterogeneous backbones break this alignment, so the shared knowledge becomes unreliable and misleads local training. We propose FedTopo, a relation‑level framework that encodes global knowledge as class relation topology, capturing how classes relate within each client rather than where they lie in feature space. Each client builds its relation topology from local prototypes and uploads it with class statistics. The server then aggregates these relations in a reliability‑aware manner that down‑weights weakly supported ones, and broadcasts the global topology to clients. The global topology guides local training by emphasizing topology‑similar negative classes. Experiments on three datasets under eight heterogeneous backbones show that FedTopo consistently outperforms parameter‑, distillation‑, and prototype‑sharing baselines, with low communication and no inference overhead. Our code is available at https://github.com/Zhaoyang‑Ma/FedTopo.
Authors:Lehan Wang, Boli Chen, Ruixue Ding, Pengjun Xie, Jinwei Huang, Zhendong Liu, Shuo Wang, Tao Lei, Xin Ouyang, Xiaomeng Li
Abstract:
Large Language Model (LLM) agents are increasingly adopted in real‑world security operations with access to host artifacts and command‑line interfaces (CLIs), making it critical to thoroughly assess their security capabilities. However, existing cybersecurity benchmarks focus on pre‑compromise settings where agents are placed in a clean and idealized environment before an attack occurs. This leaves the post‑compromise setting underexplored. To address this gap, we introduce SecRespond, the first benchmark for evaluating LLM agents on the post‑compromise incident‑response workflow. Given a forensic disk snapshot of a compromised host together with the alerts, vulnerability scans, and baseline checks reported by a host security product, agents are required to produce forensic reports on intrusions, baseline risks, and vulnerability risks, together with a remediation plan. We instantiate this task across 10 cyber ranges, each constructed from a distinct compromised cloud host, spanning 4 entry‑point types, 21 ATT&CK techniques, and 5 operating systems. We evaluate 23 frontier LLMs on the OpenCode agent harness. Experimental results show that although current agents can reliably uncover the problems exposed by alerts, they struggle to proactively investigate the disk for silent intrusions and to produce comprehensive, verified remediation plans, with no model achieving complete detection and remediation on any single range. This reveals a fundamental bottleneck in building agents for real‑world incident response. The benchmark is publicly available at https://github.com/Alibaba‑NLP/qqr/tree/main/data/secrespond.
Authors:Zezhi Liu, Zhiwei Zheng, Hanqian Luo, Deyun Qin, Shizhen Wu, Yongchun Fang
Abstract:
Temporal logic (TL) provides a compositional language for the formulation of long horizon robotic tasks, but existing TL‑conditioned trajectory generators can sidestep perception‑to‑symbol binding by encoding exact object geometry in the task graph. We introduce \emphVision‑TL‑Action, which generates action trajectories from multi‑view images, a coordinate‑free TL syntax graph, and the robot initial state. TL‑node tokens and spatial visual tokens are fused through bidirectional cross‑attention, and the resulting representation conditions a flow‑matching trajectory generator. Visual tokens are augmented only with normalized image‑plane locations and camera‑view identifiers, while a training‑only predicate‑to‑region objective encourages grounding to referenced objects. Consistent with prior work in this domain, we evaluate the model using Success@K, the fraction of tasks for which at least one of K sampled trajectories satisfies the TL specification. On Panda task, our model achieves 67.45% Success@1024, compared with 59.11% for the oracle‑state baseline. On AntMaze task, it achieves 96.35% Success@256, comparable to the oracle result of 96.88%. Resolution and intervention studies show that spatial detail depends on semantic grounding and predicate identity affects both attention and performance. These results demonstrate a direct mapping from visual observations and structured TL goals to action trajectories without requiring object geometry at inference. Code is available at https://github.com/AricLau07/vision‑tl‑action.
Authors:Zijun Lin, Zeqing Wang, Cheston Tan, Bihan Wen, Yeying Jin
Abstract:
Recent game world models can generate visually realistic and interactive environments conditioned on player actions. However, games are not defined by pixels alone; they are governed by explicit mechanics, namely state‑dependent rules that control health reduction, skill activation, and game termination. These mechanics depend on precise internal states, such as health points, skill meters, and timers, which are tightly coupled with visual observations and determine how gameplay evolves. Without modeling these state dynamics, existing game world models may generate visually plausible rollouts but violate the underlying game rules. In this paper, we propose StatePlay, a novel state‑aware game world model that jointly predicts visual content and game states to promote mechanics‑consistent generation. StatePlay adopts a mixture‑of‑transformers (MoT)‑style architecture that preserves specialized visual and state representations while enabling cross‑modal interaction, allowing predicted states to guide frame generation. Each branch is further optimized with a distinct objective suited to its modality. Experiments show that StatePlay achieves an average normalized L1 distance below 0.06 for state prediction. Furthermore, compared with models without explicit state modeling, our method improves mechanics fidelity in generated game rollouts by 18.6%. Overall, our work highlights the importance of state‑aware game world modeling and advances beyond pixel‑level realism toward complete and mechanically faithful game generation.
Authors:Yifu Liu, Raffaele Andrea Buono, Nadia Bianchi-Berthouze
Abstract:
Current Tools for Thought (TfTs) treat affect as either friction that slows cognitive progress or a signal to optimise it. Drawing on enactive cognitive science, we argue that affect is constitutive of cognition: it reshapes the trajectory of thinking, not just the speed. We identify two core barriers for Affective TfTs: the lack of Shared Attention (caring, directed attention to the user's mode of engagement) and the lack of Affective Reorienting (the capacity to use emotional moments to open new trajectories rather than reinforcing predetermined ones), and propose three design strategies that address both: Chain of Emotion X Chain of Thought, Affective Mirror, and Prompted Reorienting. The strategies are grounded in empirical findings from a study of a touch‑aware conversational agent for embodied craft learning, and are oriented as provocations for future design.
Authors:Jindong Yang, Han Fang, Weiming Zhang, Nenghai Yu, Kejiang Chen
Abstract:
Inversion‑based watermarking is a promising approach to authenticate diffusion‑generated images, yet practical use is bottlenecked by inversion that is both slow and error‑prone. While the primary challenge in the watermarking setting is robustness against external distortions, existing approaches over‑optimize internal truncation error, and because that error scales with the sampler step size, they are inherently confined to high‑NFE (number of function evaluations) regimes that cannot meet the dual demands of speed and robustness. In this work, we have two key observations: (i) the inversion trajectory has markedly lower curvature than the forward generation path does, making it highly compressible and amenable to low‑NFE approximation; and (ii) in inversion for watermark verification, the trade‑off between speed and truncation error is less critical, since external distortions dominate the error. A faster inverter provides a dual benefit: it is not only more efficient, but it also enables end‑to‑end adversarial training to directly target robustness, a task that is computationally prohibitive for the original, lengthy inversion trajectories. Building on this, we propose FARI (Fast Asymmetric Robust Inversion), a one‑step inversion framework paired with lightweight adversarial LoRA fine‑tuning of the denoiser for watermark extraction. While consolidation slightly increases internal error, FARI delivers large gains in both speed and robustness: with approximately 20 minutes of fine‑tuning on a single NVIDIA RTX A6000 GPU, it surpasses 50‑step DDIM inversion on watermark‑verification robustness while dramatically reducing inference time. Code and pretrained models are available at https://github.com/0xD009/FARI.
Authors:Kaiwen Jiang, Siya Xu, Ziyue Zhu, Chao Yang, Anh Tuan Luu, Haoran Luo
Abstract:
The rapid growth of AI workloads is turning data centers into large‑scale, volatile, yet spatiotemporally flexible grid loads, creating an urgent need for coordinated electricity‑computing scheduling. Under stringent grid constraints, schedules from general‑purpose large language models (LLMs) are often infeasible, causing line‑flow violations and unserved load. We present PowerAtlas, an LLM‑agent framework for electricity‑computing co‑scheduling that integrates historical instances, domain knowledge, and physical constraints to produce joint decisions satisfying both grid operational rules and the service‑level agreements (SLAs) of computing tasks. Working with a provincial power utility in China, we built an experimental electricity‑computing network and validated the decision loop on real data‑center data; from de‑identified operational data we further constructed ECBench, a benchmark of 2,000 scheduling instances with oracle‑optimal solutions. Experiments across eleven LLMs demonstrate the effectiveness of PowerAtlas under realistic physical operating conditions, with consistent feasibility and cost gains across three open‑weight backbones. Our code is publicly available at https://github.com/JAVA‑Jiang/PowerAtlas.
Authors:Wei-Jaw Lee, Hsuan-Yu Yeh, Ting-Yi Hu, Chih-Pin Tan, Fang-Duo Tsai, Yi-Hsuan Yang
Abstract:
Cover song generation (CSG) should preserve the melodic and linguistic content of a reference song while recreating the remaining musical components. The state‑of‑the‑art model SongEcho utilizes F_0 sequences and voiced/unvoiced (V/UV) tags for conditioning; however, implicit linguistic information from V/UV tags cannot guarantee lyric accuracy, leading to a high phoneme error rate (PER). Inspired by singing voice synthesis (SVS), we propose MPEcho, which integrates a phoneme encoder and a length regulator (LR) into the SongEcho framework. By providing explicit phoneme‑level conditioning and precise temporal boundaries, MPEcho significantly reduces PER. To enable this, we developed Phonsa, a Whisper‑based automatic transcription model that provides high‑precision phoneme‑level annotations for singing voices, overcoming the scarcity of high‑quality audio‑phoneme pairs. Experimental results validate the effectiveness of Phonsa for alignment and MPEcho for end‑to‑end CSG. The audio samples, code and weights can be accessed from https://lonian6.github.io/MPEcho.github.io/.
Authors:Weili Zeng, Yitong Xing, Fulong Liu, Chengqun Yang, Antao Xiang, Feng Tian, Jingnan Gao, Jisong Cai, Xin Wang, Xiaomin Wu, Yao Mu, Xiaokang Yang, Yichao Yan
Abstract:
World generative models are typically used through what they produce: a rendered future, a video‑conditioned action, or latent context computed by a costly generative branch. We argue that their more reusable asset is the computation that constructs a future. As a generator transforms a corrupted future into a coherent trajectory, its intermediate states organize appearance, spatial layout, and interaction across levels of abstraction. Can this future‑generative computation be internalized in a representation inferred from the present alone? We present Enfold, which transfers this computation into a representation predicted from the current visual context and language instruction. During training, multi‑level states exposed as the generator processes the observed future supervise a current‑only encoder. The learned representation is fed back to condition future generation and is read by task heads without allowing task gradients to reshape the encoder. At deployment, action prediction no longer executes the generator. Across LIBERO, RoboTwin2.0, and real‑robot tasks, Enfold supports strong control while reducing action latency by 3.7× relative to Fast‑‑WAM, Enfold‑Flash reaches 10.1×. Representation analyses show that it suppresses nuisance variation and preferentially captures changes that emerge over longer horizons. When the current scene is altered by human intervention, both the generated continuation and the executed actions adapt, which is inconsistent with fixed trajectory replay. These results recast a world generator as a source of predictive control representations: its future need not be materialized at every step if its internal structure can be enfolded into the present.
Authors:Federica Pepe, Daniele Bifolco, Costantino Martignetti, Aureliano D'Amici, Fabiano Izzo, Damian A. Tamburri, Massimiliano Di Penta
Abstract:
The responsible development and deployment of artificial intelligence (AI) systems requires rigorous documentation of their constituent artifacts, e.g., datasets, model weights, training pipelines, and runtime dependencies. Although the Software Package Data Exchange (SPDX) 3.0 standard introduced native support for AI and dataset profiles, practical tooling capable of generating standards‑compliant AI Bills of Materials (AIBoMs) in an automated and extensible manner remains scarce. This paper presents AIGen, a modular AIBoM generator that produces machine‑readable, interoperable inventories of AI system components that comply with the SPDX 3.0 AI profile. AIGen works on top of the MLflow MLOps framework and combines mining heuristics with Large Language Models to generate AIBoMs. A plugin interface allows practitioners to extend the tool with domain‑specific collectors without modifying the core codebase, supporting heterogeneous AI frameworks such as Hugging Face, PyTorch, and TensorFlow. AIGen is designed to facilitate compliance with the European Union AI Act, the NIST AI Risk Management Framework, and ISO/IEC 42001, providing a concrete, reusable foundation for transparent, accountable AI supply chain governance. Tool URL: https://github.com/danielebifolco/AIGen Tool Video: https://youtu.be/\_nAbXDWfVL4
Authors:Zeyu Wang
Abstract:
Spiking neural networks (SNNs) are promoted as an energy‑efficient substrate because sparse, event‑driven activity replaces dense multiply‑accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task. Holding architecture fixed and swapping only the hidden unit (continuous vs. leaky‑integrate‑and‑fire), plus a two‑sided target‑firing‑rate probe, we measure how far activity can be pushed down before quality breaks. Low‑load feed‑forward perception sparsifies to 5% firing at no accuracy cost; a recurrent language model cannot go below ~50% ‑‑ the recurrent state must stay active to carry information. A spiking Transformer, by contrast, sparsifies freely to 2% (3 seeds) ‑‑ so the ceiling is a property of recurrent compression, not sequence modeling. Attention escapes the floor only by storing the full key‑value cache, trading a firing floor for a memory wall: on neuromorphic hardware, recurrence and attention pay on different axes, neither escapes. We formalize the ceiling with an information‑theoretic bound rho >= H_b^‑1(log2 M / H) and confirm its predictions: the floor rises with memory load, falls with state width, and (refuting a naive memory‑only reading) rises with task difficulty. A layer‑wise input floor further caps op reduction under dense input, isolating event‑driven perception as where neuromorphic hardware wins.
Authors:Mohammed Abdullah
Abstract:
TabPFN performs classification through in‑context learning: it conditions on a set of labeled training rows (the context, or prototypes) and predicts test labels without gradient updates. On small tabular datasets, practitioners must still choose the context size and which rows constitute the context. We study how these choices affect prediction stability, accuracy, and selection cost using repeated context sampling on 15 OpenML datasets. Specifically, we investigate (i) whether larger contexts reduce prediction variability across random draws, (ii) whether accuracy depends on preserving the training distribution or on feature‑space coverage, and (iii) whether expensive selection methods such as K‑Means and farthest‑point sampling provide benefits over uniform random sampling. We find that larger contexts are both more accurate and substantially more stable, with AUC coefficient of variation decreasing from roughly 6 to 18% at k=16 to 1 to 4% at larger context sizes on datasets with room for improvement. Although accuracy correlates with distribution representativeness in random contexts, controlled experiments show that matching feature means alone can reduce accuracy by up to 0.5 AUC because it reduces context diversity. Mixed‑effects analysis identifies diversity and coverage, rather than feature‑mean matching, as the stronger predictor of accuracy (diversity beta=+0.23, p=3x10^‑12; feature‑mean shift beta=‑0.01, p=0.71). K‑Means and farthest‑point sampling achieve similar accuracy to random selection while requiring two to three orders of magnitude more selection cost. These results show that random sampling succeeds because it provides feature‑space coverage in expectation, not because it reproduces the underlying data distribution.
Authors:Tianyu Wang, Yuxuan Zhou, Wenbin Wang, Heng Li, Zikai Xiao, Junyuan Shang
Abstract:
Speculative Decoding (SD) accelerates large language model inference by allowing a lightweight draft model to propose tokens that are subsequently verified in parallel by a larger target model. Recent approaches introduce lossy verification schemes to further improve efficiency by relaxing strict distributional matching. Yet such relaxation silently rewrites the decoding distribution, and the resulting acceleration can come at the cost of unstable, sometimes severely degraded generation quality. In this work, we present a principled analysis of the distributions induced by lossy verification methods. We show that many seemingly distinct approaches differ only superficially and can be classified into two categories: truncation‑based verification and collaborative verification. We further construct a diagnostic evaluation framework across curated benchmarks. For truncation‑based methods, we identify a fundamental pitfall: performance can degrade significantly compared to the true truncation sampling baseline due to distributional distortion. For collaborative verification, we uncover a key principles: controlling the overshoot of draft probabilities relative to target probabilities is essential to prevent low‑quality outputs. Our code is available at https://github.com/ZhouYuxuanYX/Fast‑HSD.
Authors:Yuxiong Xu, Kaiqing Lin, Bin Li, Haodong Li, Sheng Li
Abstract:
Existing audio forgery detection and localization (AFDL) methods often overfit dataset‑specific low‑level artifacts, limiting their generalization to subtle, localized, and unseen manipulations. Recent audio large language model (ALLM)‑based approaches cast AFDL as question answering but still model forensic evidence implicitly, without linking manipulation cues to predictions. To bridge this gap, we propose ThinkOmni, a reasoning‑driven omni‑modal large language model that jointly performs explicit forensic reasoning, spoofing detection, and temporal manipulation localization. To enable explicit reasoning supervision, we construct Forensic‑Aware Chain‑of‑Thought (FACoT), a 100K‑sample dataset with structured forensic evidence and reasoning annotations. Leveraging FACoT, we introduce Forensic‑Aware Modality‑Incremental Learning (FMIL), which progressively aligns semantic, acoustic, and spectral‑visual representations with the LLM backbone to capture complementary forensic cues. We further propose Forensic‑Consistent Multi‑task Loss (FCML), which combines weighted cross‑entropy with an adaptive localization loss to coordinate reasoning generation, spoofing detection, and temporal localization. Extensive experiments show that ThinkOmni achieves strong cross‑dataset generalization in both detection and localization. Code, models, data, and inference examples are available at https://beyond0814.github.io/ThinkOmni/.
Authors:Georgii Kashintsev
Abstract:
Practitioner summary. Do not copy Java's threshold of eight alone: when bins grow long, hybrid‑batch (convert after load) still walks lists during insert, while hybrid‑incremental (convert as soon as a bin hits k) matches always‑tree. Prefer hybrid‑incremental or always‑tree for overloaded bins; reserve hybrid‑batch for pure bulk load then query when chains stay short after resize. Hybrid‑incremental approximates Java conversion timing, not a HashMap port. Lead metrics below are strcmp counts and heap ‑ more stable than long‑list wall‑clock. When individual hash buckets grow long, linked‑list separate chaining incurs linear per‑bucket cost. We show that when conversion runs (hybrid‑batch finalize vs. hybrid‑incremental) dwarfs the choice of threshold k for C implementers. Using one C separate‑chaining API, we compare policies under uniform‑hash FNV (including a fixed‑m probe at alpha ~ 122), forced‑bucket chaining stress, and a moderate‑load same‑API scale run (alpha = 16). Under stress, list lookup averages ~31,250 comparisons vs ~15 once treeified; mid‑load probes need ~37M comparisons under hybrid‑batch vs ~46k under hybrid‑incremental; final post‑load comparisons converge (~15). Tree buckets use about 1.7x more heap than lists. Stress wall‑clock for long lists is illustrative and run‑noisy; we therefore headline comparisons and memory. Replaying real trigram posting‑list lengths through the same policies yields the same ranking. At alpha ~ 122 without resize, some tree wins are really deferred rehash ‑ resize first when m is simply too small.
Authors:Xiaozhen Qiao, Da Zhang, Yubin Guo, Junyu Gao, Zhiyuan Zhao, Xuelong Li
Abstract:
UAV Anti‑UAV tracking is an emerging low‑altitude security task for localizing an adversarial UAV using the onboard camera of a moving observer UAV. It differs from conventional UAV tracking and ground‑based Anti‑UAV tracking because both the camera platform and the target move simultaneously. This dual‑dynamic setting induces rapid viewpoint changes, motion blur, scale variation, and visually similar distractors, making reliable appearance matching difficult. Under such rapidly changing conditions, fixed visual representations are often insufficient because target appearance becomes unreliable and feature distributions may deviate from the training domain. The target language description remains stable across frames and can therefore serve as a semantic anchor for temporal state propagation, while online feature‑distribution alignment can reduce video‑specific test‑time shifts. In this paper, we propose \emphSATATrack, a Semantic‑Aware Temporal Adaptation framework for UAV Anti‑UAV tracking. SATATrack introduces Semantic‑Aware Context Propagation (SACP), which uses the target description to guide temporal context propagation across backbone stages and preserve target identity under rapid appearance changes. An auxiliary contrastive regularizer is used during training to discourage responses to semantically similar background regions. During inference, Temporal‑Aware Distribution Alignment (TADA) aligns feature distributions online without updating model parameters, combining recent‑frame estimates with training‑time statistics for stability. SATATrack achieves state‑of‑the‑art performance on the UAV‑Anti‑UAV benchmark while remaining competitive in Anti‑UAV and UAV object tracking tasks. The code will be available at https://github.com/XiaozhenQiao/SATATrack.
Authors:Seunggeun Kim, Jaeyeon Kim, Taekyun Lee, Yuyuan Chen, Yilun Du, Sham Kakade, Sitan Chen
Abstract:
Many discrete reasoning tasks, such as code generation, are inherently non‑causal: programmers move between high‑level structure and local details, a process we call any‑order inference. For autoregressive language models, which lack a native any‑order interface, non‑causal abilities such as infilling and next‑edit prediction require hand‑designed mechanisms. Can we instead design models that natively support any‑order inference? Masked diffusion models have recently emerged as compelling candidates, as their any‑order training objective naturally offers an any‑order prediction interface. This interface, however, does not automatically yield any‑order inference. We demonstrate that this interface‑inference gap stems from positional uncertainty: fixed‑canvas, token‑level models may know what semantic component should appear without knowing where to place it. In light of this, we propose two complementary approaches: (1) Insertion‑based masked diffusion, building on FlexMDM (Kim et al, 2025), relaxes fixed‑position commitments via insertions, enabling generation across non‑contiguous regions. (2) Latent‑space masked diffusion shifts prediction to coarser semantic segments, enabling search over latent generation orders. Empirically, we train a 7B FlexMDM for Python coding and a 125M LatentMDM for GSM8K and show that both approaches induce distinct any‑order inference behaviors and improve downstream performance. We release our codebase at https://github.com/SeunggeunKimkr/genuine‑any‑order.
Authors:Kawai Chung, Chunkit Chan, Yauwai Yim, Yuxuan Liu, Haochen Shi, Weiqi Wang, Qing Zong, Tianshi Zheng, Yixuan Fu, Kai Chung Wong, Hao Liang, Yifan Gao, Xi Yang, Janet Hui-wen Hsiao, Yangqiu Song
Abstract:
Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning remains insufficiently studied. Existing evaluations predominantly examine static text or isolated visual snapshots, which do not reflect the cumulative nature of real‑world behavioral drivers. To address this gap, we introduce MultivationBench, a benchmark designed to rigorously evaluate multimodal motivation reasoning within story‑driven visual narratives. The benchmark builds upon established psychological frameworks ‑ Maslow's hierarchy and Reiss's basic desires ‑ and requires models to integrate accumulated multimodal context to infer evolving motivations. Results indicate that MultivationBench presents a significant challenge: all tested models struggle to maintain consistent motivation reasoning across sequential contexts, revealing a critical disconnect between static recognition capabilities and the dynamic reasoning essential for human‑like social understanding.
Authors:Dachuan Song, Junyu Yin, Zechen Hu, Xuan Wang
Abstract:
A known limitation of long‑context language models is their increasingly unreliable performance in non‑additive, set‑based aggregation as context length grows. Examples include cardinality estimation, set relationships, and grouped statistics, which widely exist in logs, program outputs, tables, and multi‑turn conversations. To provide the aggregation state required by these tasks, we introduce a model‑side aggregation interface that maintains compact Hash‑based HyperLogLog (HLL) sketch states alongside a frozen language model. While the model processes the context, an extractor maps each relevant record to a canonical identity. The identity is then hashed and updates the HLL state. These states can be merged across context segments and/or read out directly for downstream reasoning, avoiding an additional generate‑execute‑return cycle. We validate the proposed approach by setting the HLL state size as 2 KiB (2,048 registers), which does not increase with context length or set cardinality. In a distinct‑count experiment involving one million records, the mean relative error was 1.6%. In a separate merge test, states built from as many as 256 segments produced exactly the same readout as a single pass over the same stream. On 3,969 aggregate‑then‑reason tasks from 174 source windows, the fixed‑budget interface reached 99.2% accuracy on Gemma 4 (31B, BF16), compared with 100.0% under exact aggregation; the paired gap was 0.8 percentage points (95% window‑cluster CI: 0.5‑1.3 points). On a matched set of 174 items, our method improved over direct full‑context reasoning by 63.2 points on Qwen and 56.3 points on Gemma. The corresponding gains over chain‑of‑thought (CoT) reasoning were 60.9 and 63.2 points, respectively. On a fixed 1,200‑task Oolong‑Synth subset, our method reached 91.1% on Qwen and 99.3% on Gemma. Code is available at https://github.com/songdc98/sketchops.
Authors:Ting-Kai Hsu, Wei-Chin Wang, Kai-Xi Hong, Yu-Hua Chen
Abstract:
Guitar tablature transcription predicts the string and fret position for each note so that the resulting tablature reproduces the target musical part. Prior sequence‑to‑sequence approaches have shown promising results on large‑scale datasets, but their generalization behavior across different dataset scales remains less explored. In this work, we propose a guitar tablature transcription framework with explicit note‑event tokenization and regularized training. The proposed decoder token representation incorporates note‑event tokens together with TAB tokens, allowing note boundaries, pitch‑related events, and string‑fret positions to be represented more explicitly. We evaluate the proposed framework on DadaGP, a large‑scale dataset, and Francois Leduc, a small‑scale dataset. Our method improves tablature accuracy over the Fretting Transformer baseline on DadaGP, with especially strong gains when trained directly on the small‑scale Leduc dataset. We further introduce a pitch‑validity constrained decoding strategy that masks pitch‑invalid TAB candidates during generation rather than correcting them after decoding and simultaneously preserves the original timing and note structure from the input. This constraint improves tablature accuracy and provides a controlled setting for measuring how much error remains after pitch‑invalid predictions are removed. Our code will be released at:https://github.com/MusicGuitarTab/GuitarTab
Authors:Shayaan Siddique, Ibrahim Mian
Abstract:
The best known lower bound for the minimum Kochen‑Specker vector system in \mathbbR^3 ‑‑ 24 vectors ‑‑ rests on a computational proof whose combinatorial half emits DRAT proofs but whose geometric half does not: the non‑embeddability of thousands of candidate graphs is established by Z3's nonlinear real arithmetic, which produces no checkable proof objects. We close this gap for the proof's blocking database. We introduce exact rational case‑tree certificates of real non‑embeddability, whose splits are polynomial factorizations and rational sum‑of‑squares decompositions and whose leaves are discharged by injectivity, ideal‑membership, or Positivstellensatz‑shaped positivity arguments, and we certify all 291 source lines (180 distinct graphs) of the published pipeline's order‑10 to order‑13 blocking lists. Certificates are replayed by two independent checkers that share no code with the generator: a pure‑Python replay over exact fractions, and a total checker implemented and proved sound in Lean 4. The soundness theorem ‑‑ acceptance implies that no injective‑on‑rays, orthogonality‑respecting assignment of nonzero real vectors realizes the graph ‑‑ is kernel‑checked with axiom closure propext, Classical.choice, Quot.sound, and a gcd‑free rational arithmetic layer makes the entire verdict computation kernel‑reducible, so each per‑graph non‑embeddability result is a closed kernel theorem proved by decide. The formalization surfaced findings about the published pipeline, including a load‑bearing injectivity side condition in its embeddability notion, hidden WLOG case obligations invisible to Z3‑based workflows, and an unreproducible candidate count that we resolve against the published artifacts. All certificates, checkers, and proofs are available and replayable from a single build.
Authors:Vran Lee, Xin Liu, Yijie Wei, Yeqiang Liu, Hwa Liang Leo, Zhenbo Li
Abstract:
Tracking dense, homogeneous targets like schooling fish remains a major challenge for multiple object tracking due to extreme inter‑individual homogeneity, severe physical clustering, and rapid non‑rigid deformations. While heavy‑backbone separated detection and embedding trackers like SU‑T push accuracy boundaries using complex Re‑Identification networks, their computational overhead prohibits edge deployment. Furthermore, these modules often fail when appearance features degrade under severe occlusions. To overcome this, we propose Tracking Identities with Dual‑branch Elasticity (TIDE). Bypassing expensive appearance cues, TIDE utilizes the Adaptive Geometric Correspondence IoU, an association mechanism leveraging spatial and structural consistency to robustly handle complex morphological variations. Crucially, TIDE introduces system‑level deployment elasticity, decoupling the algorithmic pipeline from strict hardware constraints. Evaluations on the MFT‑Edge benchmark demonstrate that our Lightweight L‑branch achieves a competitive HOTA of 28.43 using merely 20.47G FLOPs. This represents a 38.7‑fold computational reduction compared to upper bounds like SU‑T, directly facilitating real‑time edge deployment. Concurrently, our Scalable S‑branch establishes a 29.98 HOTA, successfully bridging the gap between high‑precision cloud analysis and efficient edge tracking. The dataset and codes are released at https://vranlee.github.io/TIDE/.
Authors:Zheng Zhang, Nanjie Yao, Jiarui He, Deheng Ye, Peilin Zhao, Hao Wang
Abstract:
Social deduction games (SDGs) such as Werewolf have become challenging testbeds for AI agents. These games require complex social skills such as reasoning, deception, and collaboration. While recent advances in large language models (LLMs) have driven significant progress in SDG agents, current approaches are predominantly text‑based, overlooking the multimodal nature that is fundamental to human social interaction. To bridge this gap, we introduce CaM‑Wolf, the first SDG agent that integrates multimodal perception and generation. CaM‑Wolf processes video inputs from other players, employs a causal‑aware Reasoner trained via reinforcement learning to establish logical chains between observable behaviors and hidden roles, and presents itself through an animated avatar. Our experiments and user study show that CaM‑Wolf achieves superior agent gameplay performance and enhances the quality of human‑AI interaction. This work represents a significant advancement towards creating more human‑like AI agents capable of participating in nuanced social dynamics. Our code is available at https://3dagentworld.github.io/avatar_wolf.
Authors:Shaoliang Yang, Jun Wang, Yunsheng Wang
Abstract:
In a matrix‑free geometric‑multigrid FGMRES solver for three‑dimensional SIMP topology optimization, a reported converged solve is not always a converged solve. On four of 102 held‑out states, the projected residual used for stopping falls below 10^‑6 while a recomputed true residual is 1.35 to 49.5 times the tolerance; in an unguarded optimization trajectory, 22 of 40 state solves reach the iteration cap without raising an error. We formulate floor selection as a verified control problem: probe the frozen state at the original floor, use two residual features to choose the first attempted floor, and accept no solution until ||f‑Ku||/||f||\le10^‑6 is recomputed. The two‑feature rule matches 98 of 102 reference classifications; the residual guard detects the four missed escalations, and all 102 selected solves satisfy the tolerance. Relative to always using a 10^‑3 floor, the policy preserves the original operator on 24 admissible states and avoids mean compliance and gradient changes of 31.0% and 0.340 on those severe random states, and 0.48% and 0.008 on seven optimized designs, at 2.5 times the mean wall time. In a 12‑state subset of the held‑out states, eight still require escalation at the conventional floor 10^‑6. In a nine‑state control with the preconditioner's adaptive components disabled, every failure is visible and no false acceptance occurs, tying the stopping‑estimate drift to the iterate‑dependent preconditioner. The recomputed residual is the correctness safeguard; the probe and floor ladder govern an implementation‑specific cost‑fidelity tradeoff.
Authors:Loong Kuan Lee, Ragavi Krishnamoorthy, Nico Piatkowski
Abstract:
The problem of learning the graphical Markov blanket (MB) of a variable from data has applications in many areas such as structure learning for Bayesian networks and Markov random fields, causal discovery, and feature selection. However, a common assumption most methods make is that the conditional independencies in the distribution imply the same separation in the graphical structure ‑‑ also known as the faithfulness assumption. Unfortunately, this assumption can be violated by higher‑order dependencies such as XOR and parity‑type relations, and ‑‑ on finite samples ‑‑ by empirical violations that, in extreme cases, even induce spurious dependencies absent from the true distribution. Therefore, in this paper we propose a "k‑order" relaxation of the faithfulness assumption that captures parity type relationships between k+2 variables. We then propose a proof of concept algorithm called k‑order Markov blanket (kOMB) that uses this relaxation for MB discovery. Finally, we empirically show how kOMB can recover the MB of a variable under both true and empirical violations of faithfulness. Code available at: https://github.com/lklee9/k‑order‑Markov‑blanket
Authors:Mahmoud Selim, Sriharsha Bhat, Karl H. Johansson
Abstract:
Modeling and forecasting nonlinear dynamics under distribution shifts is essential for robust decision‑making in real‑world systems. In this work, we propose MetaKoopman, a Bayesian meta‑learning framework for modeling nonlinear dynamics through linear latent representations. MetaKoopman learns a Matrix Normal‑Inverse Wishart (MNIW) prior over the Koopman operator, enabling closed‑form Bayesian updates conditioned on recent trajectory segments. Moreover, it provides a closed‑form posterior predictive distribution over future state trajectories, capturing both epistemic and aleatoric uncertainty in the learned dynamics. We evaluate MetaKoopman on a full‑scale autonomous truck and trailer system across a wide range of adverse winter scenarios, including snow, ice, and mixed‑friction conditions, as well as in simulated control tasks with diverse distribution shifts. MetaKoopman consistently outperforms prior approaches in multi‑step prediction accuracy, uncertainty calibration, and robustness to distributional shifts. Field experiments further demonstrate its effectiveness in dynamically feasible motion planning, particularly during evasive maneuvers and operation at the limits of traction. Project website: https://mahmoud‑selim.github.io/MetaKoopman/
Authors:Lukas Stepanek
Abstract:
Mixture‑of‑experts (MoE) inference first aligns routed tokens into padded expert blocks, then executes packed quantized matrix multiplication over those blocks. This preprocessing is often treated as bookkeeping. In one pre‑specified Qwen3‑Coder AWQ layer‑6 fixture on a pinned vLLM/Marlin build and RTX 3090 runtime, we show that the tested route‑block interventions select exact packed arithmetic trajectories. Two fixed preconstruction histories produced distinct native alignments and exact trajectories. Injecting the opposite alignment transferred W13, activation, routed‑W2, and final outputs. Permuting two routes within one block preserved each native trajectory, while exchanging two prior‑data‑selected routes across the boundary between expert‑106 blocks 40 and 41 transferred the complete opposite trajectory. Source‑ and binary‑derived schedule geometry maps those blocks to direct/full‑K and split/global‑reduction classes. Forcing a single‑slice 200‑block grid made W13 bitwise equal. Stable canonical construction made both histories converge to a third exact trajectory. The confirmatory cohort contains 70 valid cold processes and seven required perturbation rejections. This is a causal mechanism result for one fixture, not a prevalence, allocator, portability, or serving‑impact claim.
Authors:Ads Dawson, Adrian Wood
Abstract:
Stealth, the discipline of achieving an objective without revealing your presence, capabilities, or collected intelligence, is what separates sophisticated operators from detectable ones. Elite security researchers and advanced persistent threats achieve their objectives unnoticed; autonomous agents increasingly inherit the same offensive tasks, but do they inherit the tradecraft? We introduce StealthBench,a benchmark that measures operational stealth in autonomous offensive‑security agents across six operational security (OPSEC) dimensions. We extract 11 hand‑verified OPSEC incidents from real bug‑bounty and red‑team trajectories, expanded into 14 dockerized task scenarios, where agents, despite finding real vulnerabilities, committed stealth failures inconsistent with standard operational tradecraft: embedding credentials in public uploads, deleting production resources to prove access, force‑adding uninvolved users to demonstrate a race condition. We evaluate agent trajectories using a 3‑model large language model (LLM) judge panel with majority‑vote aggregation, measuring safe success rate (solved and stealthy), Stealth@Solve (tradecraft quality among successful solves), and reckless solve rate (solved but cover blown). Our results show that no model exceeds 54% safe success rate (the compound metric requiring both task completion and stealth), confirming that OPSEC failures are systematic across model families. We release StealthBench as a public benchmark to support both the development of stealth‑aware agents and automated OPSEC monitoring for autonomous offensive‑security deployments. The interactive leaderboard, evaluation harness, and dataset are available at https://stealthbench.com.
Authors:Gaston Besanson
Abstract:
Agentic systems act, so a defect in the evidence they retrieve becomes a wrong action with a currency cost. The most dangerous enterprise defects are metadata‑borne: a stale price or a superseded record, perfectly well‑formed in the payload and betrayed only by freshness, lineage, or provenance. Such a defect never enters the agent's context, and an agent cannot doubt data it cannot see. On a priced replenishment benchmark, a competent agent silently converts an injected metadata‑borne defect into a costly action about 60% of the time, with zero data‑quality flags and behavioral doubt markers at chance (AUC <= 0.50). Across four model tiers spanning roughly 15x in inference price, the rate stays flat: capability does not buy skepticism. A metadata‑aware pre‑action gate with downstream‑only remediation recovers the loss fully on the signals its predicates cover and not at all on those they miss. A model‑free oracle derived from the task's decision geometry tracks the measured rates with MAE 0.015 (Pearson r = 0.876, interval coverage 15/16 cells), giving the flat ladder an analytical form. Evidence integrity is a systems axis distinct from model capability; mitigation depends on enforcement placement and predicate coverage. Code, frozen results, and a deterministic analysis pipeline: https://github.com/besanson/dqSarc
Authors:Nazanin Amini, Kevin Desai
Abstract:
Editing character motion often requires transferring a gesture or gait from one or more reference motions while preserving the source action, timing, root trajectory, and unselected body regions. Existing motion datasets, however, rarely provide paired targets for arbitrary part‑local content‑‑reference combinations, and self‑reconstruction training may allow a diffusion model to reproduce the content motion while underusing the routed reference. We present MoSAIC, a latent diffusion framework for part‑local reference‑conditioned motion style transfer. MoSAIC factorizes content and reference features by anatomical region, preserves the root trajectory through a separate conditioning pathway, and routes user‑selected references to individual body parts. Its central contribution is aligned intervention supervision, which constructs synchronized references and counterfactual targets through controlled local transformations, making both the requested regional response and the motion to be preserved directly observable during training. In a frozen evaluation comprising 128 motions and 896 routed conditions, part‑masked routing reduces preserved‑region error from 70.64 to 66.45~mm and matched‑noise off‑target leakage from 18.08 to 9.88~mm relative to whole‑body routing, while retaining a positive selected‑region response. A matched‑budget continuation study further shows that retaining aligned intervention supervision produces an 8.8% relative increase in selected‑target response and a 2.0‑percentage‑point increase in requested‑route influence concentration. These results demonstrate that MoSAIC improves the response‑‑preservation trade‑off required for selective and controllable part‑local motion editing.
Authors:Xuan Zhao, Jiwoong Sohn, Qinyue Zheng, Michael Moor
Abstract:
AI agents are increasingly adept at tackling complex, long‑running tasks. With the rapid surge of autonomous capabilities, human oversight is systematically lagging behind due to limited human‑centered interfacing. Aiming to address this, we introduce AgentGUI, a user‑friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long‑running sessions. AgentGUI features 1) rich agent trajectory visualizations, 2) effective manual and automated steering, and 3) integration with and coordination between open‑source and frontier agent frameworks. A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces (38% faster, p = 0.023). In a preliminary experiment, AgentGUI's automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B‑‑9B model ladder (N=50 runs per model). AgentGUI is publicly available through its project website (https://agent‑gui‑project.github.io) and open‑source repository (https://github.com/eth‑medical‑ai‑lab/agent‑gui), along with a demo video (https://youtube.com/watch?v=GSDyxN1gTF0).
Authors:Antoine Legouhy, Ross Callaghan, Yuchuan Qiao, Whitney Stee, Philippe Peigneux, Hojjat Azadbakht, Hui Zhang
Abstract:
Diffusion MRI (dMRI) relies on diffusion‑weighted echo‑planar imaging, which is highly susceptible to eddy‑current‑induced geometric distortions. These distortions vary across diffusion volumes according to gradient strength and direction, causing between‑volume misalignment that can bias downstream microstructural analyses. Current state‑of‑the‑art correction methods, such as FSL Eddy, achieve high‑quality correction through iterative prediction‑correction schemes but are computationally expensive. We propose Eddeep, a deep‑learning framework for fast eddy‑current distortion correction in dMRI. Eddeep decomposes the problem into two stages. First, a supervised image translation network standardises the appearance of diffusion‑weighted and b=0 images, removing contrast differences that hinder reliable registration. Second, an unsupervised registration network estimates both eddy‑current distortion and between‑volume head motion parameters under a physics‑constrained quadratic distortion model, enabling correction in a single forward pass. The method was trained on UK Biobank data and evaluated on both in‑domain (UK Biobank) and out‑of‑domain (Memodyn) datasets. Across a range of complementary metrics, including between‑volume jitter, diffusion kurtosis imaging residuals, signal irregularity, and mutual information, Eddeep achieved correction quality comparable to that of FSL Eddy while substantially reducing inference time. These results demonstrate that deep learning can provide accurate and efficient eddy‑current distortion correction without relying on iterative optimisation, supporting the development of faster diffusion MRI processing pipelines for large‑scale studies and clinical deployment. The code is available at: https://github.com/CIG‑UCL/eddeep.
Authors:Armin Maleki, Hayder Radha
Abstract:
Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results. In real‑world deployments, however, collaborating vehicles often use heterogeneous sensors, perception models, datasets, and training domains, creating feature‑space shifts that degrade downstream fusion and detection. Existing approaches typically retrain fusion and detection components or introduce modality‑specific feature interpreters. These methods scale poorly to newly joining agents and often require access to proprietary metadata, raising privacy concerns. We propose HeteroPROMPT, a real‑time and privacy‑preserving framework for heterogeneous collaborative perception. HeteroPROMPT rapidly aligns each heterogeneous agent's features with an ego‑centric unified feature space through modular prompts and lightweight learning‑based tuning, while keeping agent encoders and the collaborative fusion and detection stacks frozen. Its visual prompt‑based training and inference modulate Bird's Eye View (BEV) features across channels and spatial locations with low computational overhead. For metadata‑free deployment, an autoencoder learns a compact unified representation and extracts modality cues from shared features, enabling real‑time modality classification and routing to the appropriate HeteroPROMPT modules without exposing proprietary agent information. Experiments on the OPV2V‑H and V2XSet datasets show that HeteroPROMPT improves Average Precision over state‑of‑the‑art heterogeneous CP methods while using orders of magnitude fewer trainable parameters. This offers a scalable and practical CP solution. The proposed modality classifier also predicts the joining agent's modality from compact features with greater than 99.99 percent accuracy during deployment. Code will be available at https://github.com/arminmaleki007/HeteroPROMPT.
Authors:Alain Chavarri Villarello, Sander R. Dahmen
Abstract:
Number fields, which generalize the rational numbers, are fundamental objects in number theory. Many of their key arithmetic properties are captured by invariants whose computation is among the central tasks of computational algebraic number theory and a focus of several computer algebra systems and databases. In this paper, we describe a Lean 4 formalization for certifying several of these number field invariants. Building on previous work on certifying rings of integers, we extend this certification approach to further invariants including the signature, the unit group modulo p‑th powers, and, ultimately, the class group. We also improve discriminant certification, allowing verifications for higher‑degree number fields infeasible in previous work. We introduce structures based on representations of algebraic objects suited to computation, including reusable ones for certifying ideal arithmetic. Along the way, we formalize several underlying mathematical results, for instance on real closed fields and pseudo‑remainder sequences, which are of independent interest. We apply our framework to verify hundreds of entries of the L‑functions and modular forms database (LMFDB) concerning the discriminant, signature, class number, and class group structure of various number fields. To this end, we wrote a SageMath script that computes the certificates and outputs Lean proofs of the corresponding statements.
Authors:Siqi Zeng, Sewoong Lee, Han Zhao, Julia Hockenmaier
Abstract:
Instruction hierarchies are a core safety assumption of language model deployment: higher priority inputs, such as system prompts, should override conflicting lower priority inputs from users or tools. Yet frontier LLMs often violate this hierarchy. We introduce V‑Steer, a training‑free inference time method that restores privileged influence by editing cached value vectors at prompt positions. Using direct logit attribution on the first next token prediction, V‑Steer identifies heads where lower priority spans dominate privileged ones, then boosts privileged spans and suppresses conflicting lower priority spans through in‑place multiplicative edits to cached V tensors. Since the method acts only on cached values, it remains compatible with fused attention backends and adds only a one time prefill overhead. Across models from 7B to 70B, this attribution guided intervention raises primary constraint accuracy from under 18% up to 92% on controlled role conflict benchmarks, and on broader instruction hierarchy evaluations substantially outperforms prompt only baselines while matching or exceeding SoTA training based methods on 3 of 4 scales of LLMs, with negligible decoding‑speed overhead. The code is available at https://github.com/cindy2000sh/v‑steer.
Authors:Mengya Hu, Susie Park, Suzana Ilic, Qiong Wei, Sandeep Atluri, Myra Deng, Tucker Fross, Curt Tigges
Abstract:
Content‑moderation classifiers are usually evaluated in isolation, but deployment requires choosing where to intervene and what follows a flag. We evaluate these choices using two end‑to‑end customer‑outcome metrics rather than component accuracy: Usefulness, the fraction of turns with a shown, non‑harmful, relevant response, and Harmful Exposure, the fraction with a shown harmful response. Latency and error rates are diagnostics. We compare Input only, Response only, and Input + response hard blocking on a human‑labelled product benchmark and public ToxicChat evaluation. At the evaluated operating points, Response only achieves the highest filter‑only Usefulness in both settings, while Input + response achieves lower Harmful Exposure. Replacing Response only blocking with Response + rewrite recovers most blocked traffic and yields the same observed Harmful Exposure count as Response only blocking for the selected configuration; this equality is not an equivalence result. Probe routing substantially reduces conditional route‑and‑generation time relative to LLM routing at comparable measured outcomes. A focused output review shows how rewrites balance filter passage with usefulness by generalizing triggering language while retaining benign intent and safe redirection; some sensitive‑domain outputs nevertheless omit potentially safety‑relevant support information. These results support comparing moderation configurations under deployment‑specific safety and latency constraints rather than applying a universal placement rule. Code and public artifacts are available at https://github.com/microsoft/mod‑frontier
Authors:Yung-Hsu Yang, Luigi Piccinelli, Siyuan Li, Mattia Segu, Lei Ke, Martin Danelljan, Yuqian Fu, Zuria Bauer, Fisher Yu, Hermann Blum, Marc Pollefeys
Abstract:
Safe autonomous navigation requires a holistic understanding of dynamic environments, necessitating the simultaneous estimation of metric depth, semantic segmentation, and instance trajectories. While depth‑aware video panoptic segmentation (DVPS) unifies these tasks, existing approaches often rely on computationally expensive, multi‑stage pipelines or offline tracking, rendering them unsuitable for real‑time decision‑making. To address this, we propose DVPSFormer, a unified online architecture designed for efficient 4D scene understanding. Central to our approach is explicit scene discretization (ESD), a novel mechanism that leverages segmentation queries to represent foreground and background regions, enabling a discrete‑to‑continuous (D2C) depth head to decode metric depth in a single pass. This tightly couples semantic and geometric learning while significantly reducing latency. Furthermore, we propose an online majority voting (OMV) mechanism that exploits temporal consistency to refine classification during instance tracking. DVPSFormer establishes a new state‑of‑the‑art on the Cityscapes‑DVPS and SemKITTI‑DVPS benchmarks, offering a streamlined solution for online robotic perception. Code and models are available at https://royyang0714.github.io/DVPSFormer.
Authors:Brett Reynolds
Abstract:
An AI benchmark result rarely reaches a consequential claim in one step. Evaluators generalize it to further cases, interpret it as evidence of capability, extrapolate it to new tasks, transport it to another system or site, and combine it with assumptions about human review and downstream consequences. Validity‑centred approaches require evidence for each claim. This paper identifies a further epistemic problem: warranted links don't automatically make a warranted chain. The target of one study may not be the source of the next; system, population, outcome, or conditions may change at the interface; and shared data or model lineage may make apparently independent support dependent. Projectibility concerns whether a bounded extension from observed to unobserved cases is warranted. Goodman supplies the problem of rival extensions; argument‑based validity supplies an architecture for testing them. The paper's distinctive claim is a non‑composition principle: support for adjacent projections warrants their composition only when endpoints and assumptions align and dependence and uncertainty are carried through. A legal‑research case shows how benchmark evidence and a deployment study can each be sound while remaining parallel. A reanalysis and simulation show why aggregate stability can erase distinctions a later projection requires. The resulting projectibility audit diagnoses unsupported joins in benchmark‑to‑use arguments.
Authors:Yuvraj Verma
Abstract:
Self‑repair ‑ returning a failed program to the model together with its test output and asking for a correction ‑ is a standard component of code agents, and is almost always evaluated against a baseline that does not retry at all. We argue that this comparison confounds the value of the feedback with the value of the extra attempt. Using a placebo‑controlled design on MBPP+ at three model scales (1.5B, 3B, 7B), we compare four matched‑budget retry conditions: blind resampling, a content‑free failure notice, genuine execution feedback, and feedback augmented with verbal self‑reflection. Blind resampling is the strongest condition below 7B, and remains statistically tied with the best condition at 7B, while consuming 2.5‑5.5x fewer tokens; conditioning on the model's own failed attempt costs 6.1 points at 1.5B (p=0.006), and the informational content of execution feedback adds nothing measurable over the placebo. We attribute this to anchoring: when shown its previous attempt, a model reproduces a near‑identical program in 33‑68% of retries, against 2‑14% under blind resampling. Two further experiments delimit the effect. Retrieved solutions to other tasks change nothing (bounded to +/‑3.5 points), which localizes the harm to self‑conditioning rather than context length; and reflection, the only condition that measurably weakens the anchor, remains dominated on cost. Replication rules out two competing explanations: the penalty is unchanged at full precision, and it reproduces on an independent model family. Across six configurations spanning two families and two precisions, its magnitude is predicted by baseline quality alone (r=0.96) ‑ the cost of anchoring is the cost of committing to a bad first attempt.
Authors:Nicholas T. Runcie, Fergus Imrie, Charlotte M. Deane
Abstract:
Systematic International Union of Pure and Applied Chemistry (IUPAC) names are standard for communicating molecular structures in chemical literature, patents, and databases. We introduce NISPO, an open‑source RDKit‑based Python package for IUPAC name generation. NISPO was developed by an agentic self‑improvement loop using OpenAI's Codex with the GPT‑5.5 model. A generated name was considered correct if the open‑source OPSIN tool parsed it back to the input structure. Guided by this objective, the agent implemented and refined NISPO against 2.68 million molecules from SureChEMBL, resulting in the tool achieving 98.1% round‑trip accuracy on a held‑out set of 103 million PubChem molecules. NISPO is freely available at https://github.com/oxpig/nispo.
Authors:Penglong Zhai, Bowen Zheng, Jie Li, Yifang Yuan, Yue Liu, Sicong Wang, Mingyang Yin, Tingting Hu, Shuaijun Guo, Fanyi Di, Xin Li
Abstract:
Generative retrieval enables recommender systems to retrieve items by generating compact item identifiers, but scaling it to industrial scenarios remains challenging due to redundant or colliding token assignments and insufficient integration of heterogeneous item signals. These challenges are particularly critical for next Point‑of‑Interest (POI) recommendation, where models must represent structured spatial entities, capture sequential mobility patterns, and produce predictions consistent with real user behavior. We propose Gwhere, an end‑to‑end industrial framework that integrates semantic identifier (SID) generation with LLM‑based generative next POI recommendation. Gwhere first learns discriminative POI SIDs through a contrastive residual‑quantization tokenizer that aligns textual, visual, spatial, and collaborative signals. Based on these SIDs, Gwhere adapts LLMs to mobility scenarios via continued pretraining on enriched spatio‑temporal corpora, supervised fine‑tuning, and Exposure‑Aware Kahneman‑Tversky Optimization (EAKTO), a reinforcement learning objective for behavioral preference alignment. Experiments on public datasets and Amap's large‑scale industrial dataset demonstrate the effectiveness of Gwhere. The system has been deployed in Amap's homepage service under high‑concurrency and low‑latency constraints. Long‑term online A/B tests show improvements of 5.83% in P‑CTR and 6.20% in U‑CTR over the production baseline. The implementation is publicly available at https://github.com/alibaba/SimCIT.
Authors:Guanming Xiong, Penghui Zhang
Abstract:
Large language model (LLM)‑based agentic search systems are often evaluated as if the underlying LLM were the only component that matters, yet their measured performance also depends on the surrounding search environment: the Wikipedia snapshot, preprocessing pipeline, chunking policy, retrieval backend, tool schema, observation format, and answer submission rule. These details are frequently under‑specified, making it difficult to compare results or reproduce reported baselines. We present SimpleWikiSearch, whose corpus construction, retrieval stack, tool contract, and evaluation protocol are explicit and runnable. The environment starts from a full English Wikipedia dump, cleans and chunks the corpus, builds keyword and dense retrieval indexes, and exposes a minimal tool interface consisting of \textttsearch, \textttopen\_url, and \textttsubmit\_answer. We report baseline results on six QA datasets using open‑source LLMs and provide a random‑300 subset for comparisons with closed‑source commercial models. SimpleWikiSearch provides a domain‑specific agent harness and a controlled offline environment for reproducible agentic‑search evaluation. Its contribution is this specified reference setup, rather than a new agent algorithm. Code and data will be available at: https://github.com/JimXiongGM/simple_wiki_search.
Authors:Ranjitha Shivaprasad Ballakuraya, Arash Mahyari, Ashok Srinivasan
Abstract:
The growing volume of scientific submissions has motivated interest in using large language models (LLMs) to assist peer review. Existing automated novelty assessment approaches typically compare a paper's claimed contributions against prior literature, implicitly assuming that these contributions are accurately realized in the work itself. Human reviewers, however, frequently challenge novelty claims not because similar ideas already exist, but because the methodological evidence presented in the paper does not adequately support them. This internal mismatch between claimed contributions and methodological realization is rarely examined by current LLM‑based review systems. To address this gap, we introduce intra‑paper claim verification, a framework that evaluates whether novelty claims articulated in a paper are substantiated by the methods used to realize them. The framework employs an LLM to extract novelty claims from the introduction, retrieve claim‑relevant methodological evidence, and assess whether the methods substantiate the stated contributions. Assessment is guided by reviewer‑inspired evaluation criteria derived inductively from human peer reviews collected from 182 ICLR 2025 papers. These criteria capture recurring reviewer concerns related to novelty, methodology, clarity, and other issues and are used to generate structured reviewer‑style assessments of claim substantiation. We evaluate the framework by comparing LLM‑generated review comments against human reviewer concerns on a balanced subset of accepted and rejected papers. Human evaluation demonstrates significant alignment between framework‑generated assessments and human reviewer concerns, particularly for novelty‑related issues. BERTScore further distinguishes corresponding human‑LLM review pairs from mismatched controls, indicating that the framework captures concerns consistent with human reviewer observations.
Authors:Mouad Zemzoumi, Amine Abouaomar
Abstract:
Pre‑match tactical decision‑making in professional football relies heavily on subjective expert analysis and identity‑based scouting systems that cannot generalize to unseen teams. This paper presents Sim2Win, a team‑agnostic, event‑based pre‑match tactical recommendation framework that reframes match outcome prediction as a tactical decision‑support problem. Using StatsBomb open event data from eleven competitions spanning 178 teams and 1,411 team‑match records, Sim2Win constructs five‑match rolling tactical profiles, engineers four interpretable tactical feature ratios, clusters team behaviors into eight playstyles via K‑Means, and trains thirteen classifiers to estimate win, draw, and loss probabilities from tactical matchup representations. The system operates without team names or identity features, enabling generalization to teams never seen during training. A rigorous Leave‑One‑Competition‑Out (LOCO) evaluation demonstrates that Sim2Win achieves a mean ROC‑AUC of 0.704 and mean accuracy of 55.4% on completely unseen teams, outperforming ELO, Pi‑Rating, and GAP baselines on all 21 ROC‑AUC comparisons and 19 of 21 accuracy comparisons. Among all evaluated models, CatBoost achieved the strongest in‑distribution performance with 60.90% accuracy. These findings suggest that behavioral tactical representations provide transferable predictive signal under distribution shift and offer a viable alternative to identity‑dependent football prediction systems.
Authors:Haolei Xu, Xiaowen Xu, Haiwen Hong, Zixuan Ni, Hongxing Li, Yiwen Qiu, Weiming Lu, Yongliang Shen
Abstract:
On‑policy distillation (OPD) grounds token‑level supervision in the student's own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher‑student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label‑free handoff trigger in Relay On‑Policy Distillation (Relay‑OPD). During training, Relay‑OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qwen3‑4B‑Instruct‑2507 teacher and Qwen3‑0.6B/1.7B‑Non‑Thinking students on eight mathematical reasoning benchmarks, Relay‑OPD achieves the best or second‑best results on every benchmark, outperforming standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for 1.7B, with consistent gains at 0.6B. Training trajectory length is reduced by over 50%.
Authors:Kaneyoshi Hiratsuka, Benjamin Yen, Ryosuke Kojima
Abstract:
Acoustic information provides rich cues about object location, material properties, and changes caused by contact or motion. This paper introduces a new set of acoustic‑aware manipulation tasks for imitation learning, in which robots must use auditory cues to determine manipulation targets. These tasks require sound source localization and identification for active exploration in robotic manipulation. Also, we propose a multimodal imitation learning framework, Spatial‑Spectral Audio Action (S2A2), that integrates visual features with acoustic spatial and acoustic signal information for the acoustic‑aware manipulation tasks. We implemented S2A2 models that integrates policies such as ACT, Diffusion Policy, VQ‑BeT, and π_0, into our framework. Simulation experiments showed that the proposed method is the most effective for tasks requiring both position and timbre. Furthermore, real‑robot experiments confirm the applicability of the proposed tasks and framework to real‑world manipulation.
Authors:Yuan Yin, Elias Ramzi, Marc Lafon, Valentin Charraut, Victor Bares, Yihong Xu, Éloi Zablocki, Alexandre Boulch, Thibault Buhet, Andrei Bursuc, Matthieu Cord
Abstract:
Self‑play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera‑input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective‑view self‑play as a practical training regime. We introduce Pictura, a GPU‑accelerated multi‑agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent‑steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self‑play with plain PPO. It is the first large‑scale driving self‑play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero‑shot to Waymo Open Motion Dataset layouts re‑rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/
Authors:Timy Phan, Jannik Wiese, Björn Ommer
Abstract:
Predicting how a scene may evolve from partial observations requires reasoning about multiple possible futures rather than committing to a single trajectory. Existing approaches either generate appearance‑dominated video predictions or sample a small number of trajectories without explicitly modeling the distribution of possible motion. We introduce Goal‑Aware Representations of Future kInEmatic Latent Distributions (GARFIELD), a probabilistic model of scene kinematics that learns a structured spatio‑temporal latent representation of the distribution over possible futures given an image and optional spatio‑temporally sparse constraints. The same latent representation enables both joint sampling of all trajectories and direct access to the underlying motion distribution through an efficient deterministic density decoder. As a result, uncertainty about future motion can be localized to specific scene elements and timesteps and progressively refined through additional constraints. Experiments demonstrate strong motion planning performance competitive with large video generation models while sampling trajectories 97× faster. Our method further estimates motion densities two orders of magnitude faster than Monte‑Carlo sampling from motion generation models, enabling interactive exploration and uncertainty‑aware planning.
Authors:Fanfu Wei, Thibault Ehrhart, Raphaël Troncy
Abstract:
Wikipedia and Wikidata are widely used for information access, LLM pre‑training, and retrieval‑augmented generation. Their knowledge is deeply connected but scattered across text, tables, and knowledge graphs. This raises a practical question: when these modalities disagree, how can we detect and explain the conflict? We study this problem as \emphmodality‑level inconsistency detection. We first introduce a taxonomy of cross‑modal knowledge inconsistencies, covering information granularity differences, direct conflicts, temporal changes, and KG incompleteness. We then present \textscKontrast, an automatic framework that uses Text‑to‑SPARQL and LLM reasoning to compare table‑based answers with KG evidence and categorize the resulting inconsistencies. Experiments on various Table‑QA datasets show that cross‑modal inconsistencies are common and informative. They reveal not only true knowledge conflicts, but also missing KG structure and temporal mismatches while being limited by Text‑to‑SPARQL errors and noise. Our analysis shows that text, tables, and KGs can complement and correct one another through systematic comparison. \textscKontrast provides a practical tool for large‑scale knowledge auditing and establishes a benchmark for future work on cross‑modal knowledge consistency. Code and data are available at https://github.com/ECLADATTA/KONTRAST.
Authors:Hui Wei, Hao Yu, Guoying Zhao
Abstract:
Face de‑identification (De‑ID) aims to remove or conceal personally identifiable facial features in images or videos to prevent identity recognition while preserving utility for downstream tasks. With the rising emphasis on data privacy and responsible AI, face De‑ID has emerged as an active research area spanning computer vision and privacy‑preserving communities. Early approaches, and many contemporary ones, operate in the digital domain by modifying pixel‑level or appearance‑level features through post‑capture processing. Recent advances extend face De‑ID beyond post‑processing by integrating privacy mechanisms directly into sensors during image acquisition, bridging sensing systems and downstream vision algorithms. In parallel, physical‑domain methods explore wearable accessories and materials that conceal identity information in real‑world environments prior to capture. In this survey, we present the first unified overview that spans the full data acquisition pipeline, encompassing the physical, sensor, and digital domains. Through this domain‑centric lens, we systematically analyze current methodologies, technical progress, and the distinct challenges inherent to each stage. We then review and organize existing evaluation protocols, examining current practices and highlighting the critical need for standardized, comprehensive benchmarks. Finally, we identify key open problems and outline emerging research directions to guide future work in this rapidly evolving field. To support ongoing research, we maintain a project page that organizes relevant literature with collected datasets and open source code: https://github.com/CV‑AC/Awesome‑FaceDe‑ID.
Authors:Fanqing Meng, Lingxiao Du, Qiguang Chen, Ziqi Zhao, Haocheng Lu, Mengkang Hu, Michael Qizhe Shieh
Abstract:
Recursive self‑improvement requires turning evidence of model failures into better models. Data‑centric post‑training research entails diagnosing capability gaps, designing and validating training‑data strategies, and learning from checkpoint feedback. Can LLM agents automate this loop? Existing benchmarks entangle research decisions with optimization, serving, evaluation, and systems implementation, obscuring agents' research capability. We introduce RSIBench‑Data, a controlled benchmark of LLM agents as data‑centric researchers with a fixed post‑training stack. Agents iteratively revise training‑data strategies for a fixed target model; training and serving use Tinker‑backed services, official evaluation runs through Harbor and E2B sandboxes, and budgets are fixed across agents. We evaluate four frontier agents on six benchmarks across software engineering, terminal use, scientific question answering, and mathematics. Agents demonstrate core data‑centric research capabilities: in 58.33% of settings, they improve upon the first valid attempt by refining strategies from feedback. However, improvement is inconsistent. Among searches continuing after the best observed score, 78.26% end with a lower‑scoring final attempt, while the rest only recover the same peak. A strong candidate may therefore appear early or midway through a run even as later revisions fail. Trajectory analysis identifies four patterns in stronger runs: accurate hypotheses, validation‑grounded supervision, behavior‑aligned data, and preservation of strong checkpoints. These findings suggest that current agents can make useful data‑centric discoveries but cannot yet translate feedback into consistent improvements. RSIBench‑Data provides a measurable, auditable testbed for the research capabilities required for recursive self‑improvement. We open‑source our code at https://github.com/evolvent‑ai/RSIBench‑Data.
Authors:Du Yin, Xiachong Lin, Yue Tan, Jinliang Deng, Estrid He, Hao Xue, Flora D. Salim
Abstract:
Traffic forecasting is important for efficient traffic management and route planning in smart cities. Existing traffic forecasting studies typically assume fixed sensor graphs, overlooking the continuous evolution of real‑world traffic networks, e.g., ongoing road network construction and evolving human mobility patterns. These dynamic changes can substantially degrade conventional forecasting models, motivating test‑time adaptation (TTA) to efficiently adapt pretrained models during deployment. However, applying TTA to evolving traffic sensor networks remains challenging in two aspects. First, topology expansion introduces new sensors and connections, continuously reshaping the sensor graph. Second, tem‑ poral shifts vary in time scale and stability, requiring differentiated adaptation to long‑term and short‑term shifts. In this study, we address these challenges by proposing A2TTA, an Anchored‑and‑Agile Test‑Time Adaptation framework for evolving traffic sensor networks, which transforms topology‑induced forecasting errors into an expandable output calibration problem and separates tem‑ poral adaptation into persistent global correction and agile context‑specific specialization. By jointly addressing topology evolution and multi‑scale temporal shifts, A2TTA enables efficient and robust adaptation to continuously evolving traffic environments. Extensive experiments on ten real‑world traffic networks demonstrate that A2TTA consistently improves forecasting performance across different backbones, datasets, and prediction horizons. Our code is available in https://github.com/lixus7/A2TTA.
Authors:Yu Hao, Jinxuan Cai, Qi Zhang, Yawen Li, Zhiqiang Zhang, Chuan Shi, Cheng Yang
Abstract:
Skills have become an important abstraction for enabling large language model (LLM) agents to reuse past experience in long‑horizon interactive tasks. However, existing trajectory‑to‑skill methods often produce flat collections of high‑level textual skills that are stored and retrieved independently, leaving skill relations underutilized and maintaining a gap between high‑level skills and executable actions. In this paper, we propose HiSkill, a hierarchical skill graph framework that organizes interaction trajectories into a directed graph with skill nodes, AtomicOp nodes, and typed edges. Specifically, the graph connects reusable high‑level skills with executable action templates, while also capturing decomposition, temporal transition, compatibility, support, and recovery relations among them. At inference time, HiSkill retrieves a compact task‑relevant subgraph and performs subgraph‑guided task execution, where a symbolic task state, an active skill, and the retrieved subgraph guide the LLM agent to switch skills, select AtomicOps, and ground executable actions iteratively. Experiments on three interactive environments show that HiSkill outperforms state‑of‑the‑art baselines while reducing inference token consumption, demonstrating the effectiveness of bridging high‑level skills and executable action grounding through a hierarchical skill graph. Our data and code is available at https://github.com/BUPT‑GAMMA/HiSkill.
Authors:Varad Shinde, Nikhil Kumar Shrey, Magesh Rajasekaran, Md Saiful Islam Sajol, Harshil Bhargava, Subhajit Sidanta, Supratik Mukhopadhyay, Yimin Zhu
Abstract:
Deep learning models in computer vision face significant challenges when trained on long‑tailed datasets, where a few majority classes dominate while many minority classes are severely underrepresented. Such imbalances frequently arise in real‑world scenarios such as rare species recognition, manufacturing fault detection, and medical image understanding, leading to biased models that underperform on tail classes. Existing reweighting methods typically rely on static class frequencies to penalize the model, ignoring the dynamic nature of how effectively a network actually learns a class over time. We address this by introducing a novel Learning‑Dynamics Aware Loss (LDAL) function that shifts the focus from static sample counts to dynamic learning progress. LDAL framework adjusts class weights continuously by leveraging: (i) the strength of learned feature representations (semantic scale), (ii) the intrinsic learning difficulty of each class, measured via the Shannon entropy of its predictions, and (iii) an inter‑epoch regularizer term that tracks prediction shifts between consecutive epochs to stabilize training and avoid local minima. LDAL is purely a objective function which incurs negligible computational overhead while adapting to the feature learning of the model. Experimental results on multiple benchmark datasets demonstrate that our approach significantly surpasses state‑of‑the‑art reweighting loss functions, providing an optimal trade‑off between accuracy and generalizability. The source code is available at https://github.com/sdm2026/ldal
Authors:Tresor Y. Koffi, Youssef Mourchid, Yohan Dupuis
Abstract:
Falls represent a critical public health challenge, and accurate detection of the impact moment when an individual hits the ground is crucial for timely intervention. Existing skeleton‑based methods rely on graph neural networks modeling only pairwise joint connections, failing to capture multi‑joint coordination characteristic of fall impacts, while transformer‑based temporal models suffer from quadratic complexity limiting real‑time deployment. We propose FLASH, a novel framework integrating single‑matrix hypergraph representations with Mamba's selective state‑space models through adaptive feedback mechanisms for efficient impact detection. Our approach constructs biomechanically‑grounded hyperedges to model functional joint coordination while leveraging Mamba's linear‑time complexity to capture temporal dynamics. Experiments on UP‑Fall and UMAFall datasets demonstrate that FLASH achieves state‑of‑the‑art accuracy with real‑time inference capability and strong zero‑shot cross‑dataset generalization, while significantly reducing computational cost compared to dual‑representation and transformer‑based methods. The model provides interpretable feedback through learned attention patterns aligned with biomechanical principles. Code is available at https://github.com/Tresor‑Koffi/FLASH‑Impact‑Fall‑Detection.
Authors:Songtianhao Xu, Zhongwei Chen, Zhao-Xu Yang, Weifeng Wang
Abstract:
Most existing drone‑view geo‑localization (DVGL) benchmarks contain drone imagery captured under a single illumination condition and lack geographically aligned visible drone images, infrared drone images, and satellite images from the same locations. To evaluate the generalization capability of DVGL methods under challenging illumination conditions, some methods train models on a visible benchmark and test them on an independent infrared benchmark. This protocol essentially constitutes transfer between datasets, which makes it difficult to systematically evaluate DVGL across daytime and nighttime conditions within a unified benchmark. To address this limitation, we construct IRCHN,a real‑world DVGL benchmark designed for localization across different illumination conditions. IRCHN contains 26,460 images collected from 8,820 geographic locations across four representative scene categories, including farmland, coastline, forest, and urban areas. Each location provides one visible drone image, one infrared drone image, and one corresponding satellite image, which enables unified evaluation of DVGL methods across different illumination conditions and sensing modalities. We further propose the Modality‑Adaptive State‑Space Transport Relation Network (MASTR‑Net), a DVGL framework tailored to localization under varying illumination conditions. MASTR‑Net integrates modality‑adaptive feature enhancement, bidirectional selective state‑space relation modeling, and soft optimal transport relation alignment to jointly reduce modality gaps and view‑induced structural discrepancies. Extensive experiments demonstrate that MASTR‑Net outperforms existing state‑of‑the‑art methods on IRCHN for localization under varying illumination conditions and achieves competitive performance on two infrared benchmarks, IR‑VL328 and CVGL‑RGBT. Code: https://github.com/SongtianhaoXu/MASTR‑Net
Authors:Hao Liang, Meiyi Qiang, Sizhe Qiu, Linzhuang Sun, Wentao Zhang
Abstract:
Enterprise agents often need to integrate heterogeneous knowledge sources: documents for narrative facts, tables for computation, and dependency graphs for file relationships. Existing benchmarks typically evaluate retrieval or tool use without distinguishing whether an agent first selects the appropriate knowledge sources. We introduce WorkSurface‑Bench, a benchmark for evaluating this capability as surface routing. It contains 1,151 atomic tasks derived from persona‑scoped Workspace‑Bench‑Lite workspaces, spanning document, table, graph, and cross‑surface questions. Its reference answers are auditable: table answers are reproduced through executed DuckDB queries, document answers are grounded in verified text spans, and graph answers are traced to source dependency annotations. We evaluate four model backbones across six controlled agent settings, yielding 27,624 protocol‑error‑free trajectories. Under gold‑constrained tool access, agents achieve 98.7‑99.8 Route F1, while Answer remains only 56.1‑75.3 percent, showing that correct surface selection is necessary but insufficient for task completion. Matched interventions further show that surface hints improve Answer for three of four models, whereas removing irrelevant tools primarily improves routing and efficiency. In an independent three‑annotator audit, all 200 sampled tasks pass all six quality criteria by majority vote, with 192 receiving unanimous judgments on every criterion. We release the dataset, construction pipeline, scoring code, and agent harness at https://github.com/haolpku/WorkSurface‑Bench.
Authors:Qi Chen, Siria Xiyueyao Luo, Jian Wang, Yuan Shi, Haocong Rao, Xuejiao Zhao
Abstract:
Cognitive distortion amplifies negative emotions and contributes to mental health disorders. Cognitive Behavioral Therapy (CBT) is an effective way to address cognitive distortions, but its large‑scale application is limited by the shortage of professional therapists. Although large language models (LLMs) have recently been explored for mental health applications, existing methods still suffer from limited domain specificity, overly flattering responses, and the absence of well‑defined annotations for cognitive distortions. This paper proposes Cognivia, an evidence‑based artificial intelligence therapist that integrates automatic cognitive distortion identification and rational response generation. Our framework is built on authoritative CBT texts widely regarded as core paradigms and standard references. It is further augmented with mental health question‑answer (Q and A) data, and employs multi‑stage prompting and structured generation strategies under the supervision of behavioral science experts. Then we fine‑tune a lightweight LLM on this augmented CBT dataset to obtain Cognivia. In addition, we propose the first hierarchical quality evaluation framework for assessing LLM‑generated rational responses, developed through collaboration between AI researchers and behavioral science experts. Cognivia is evaluated using lexical metrics, LLM‑based Judges with two complementary criteria, and human evaluation by 10 behavioral science experts. It consistently outperforms the baseline methods in cognitive distortion recognition and rational response generation, demonstrating its effectiveness. Our code is available at https://github.com/SNOWTEAM2023/Cognivia.
Authors:Rebecca Ramnauth, Brian Scassellati
Abstract:
Transformer adaptation is typically distributed across model depth, even when the intended change is narrow. We investigate how adaptation site shapes what a model learns, how well that learning generalizes, and how selectively it is applied. We introduce a controlled benchmark spanning five objectives (lexical binding, factual association, behavioral policy learning, causal mapping, and procedural reasoning) and define each objective's "adaptation geometry" as its profile of acquisition, transfer, and boundedness under full‑stack and early‑, middle‑, or late‑layer LoRA. The objectives exhibit distinct geometries. Lexical binding favors early‑layer adaptation for acquisition and boundedness but requires broader updates for transfer; factual association favors later layers among localized adapters; behavioral learning separates late‑layer action acquisition from middle‑layer policy gating; and causal and procedural transfer benefit most from middle‑ or full‑stack adaptation. These patterns largely persist under parameter‑matched controls, and most corresponding directional contrasts replicate across five model families. These findings establish adaptation site as a key design variable for controlling what models learn, generalize, and leave unchanged.
Authors:Yajing Xu, Yarong Lan, Jiaoyan Chen, Yichi Zhang, Jeff Z. Pan, Mingchen Tu, Zhizhen Liu, Wen Zhang, Huajun Chen
Abstract:
While text‑to‑image models exhibit remarkable visual fidelity, they frequently violate fundamental physical commonsense. Existing benchmarks often rely on coarse‑grained descriptions, failing to diagnose the mastery of specific physical principles. Moreover, the high stochasticity of generative processes causes current prompt optimization methods to suffer from gradient hallucinations, where optimizers are misled by transient visual artifacts rather than systemic flaws. To address these challenges, we introduce OmniPhys, a rigorous benchmark of 1,551 samples grounded in a Physical Knowledge Graph. By aligning PhET simulations with standard curricula, OmniPhys operationalizes a knowledge‑to‑scenario pipeline that performs diagnostic stress tests via a dual‑path verification protocol. We further propose OmniPrompt, an iterative framework that treats physical alignment as a discrete optimization problem. For each query, OmniPrompt aggregates K stochastic images into a per‑query feedback buffer. Across training, it further merges feedback from batches of B queries before each meta‑policy update, filtering seed and query‑local noise. Evaluations across 12 representative text‑to‑image models reveal universal physical bottlenecks. Results demonstrate that OmniPrompt significantly enhances physical consistency across diverse backbones, proving the transferability and efficacy of our evolved meta‑policies. The code and data are available at https://github.com/zjukg/OmniPhys
Authors:Ibrahim Mian, Shayaan Siddique
Abstract:
The Erdős‑Selfridge odd covering problem (Erdős problem #7) asks whether a covering system of \mathbbZ exists whose moduli are all odd, distinct, and greater than 1. The problem is open. We present a Lean 4 formalization, checked end to end by the proof kernel, of the exclusion: any covering of \mathbbZ by finitely many congruence classes with distinct odd moduli > 1 has lcm of the moduli exceeding 10000. The proof composes a formalized density argument (a covering by divisors of N exceeding 1 forces 2N \le σ_1(N), so the lcm is abundant or perfect), a kernel‑checked abundancy floor (no odd N < 945 qualifies), a family of Chinese‑Remainder capacity certificates ‑‑ decidable per‑N arithmetic inequalities each refuting every covering with distinct moduli > 1 dividing that N ‑‑ for all 23 odd abundant numbers below 10^4, and a kernel‑checked enumeration establishing that those 23 are the only odd non‑deficient candidates. The result is transported to the official StrictCoveringSystem \mathbbZ formulation of Erdős #7 in google‑deepmind/formal‑conjectures, with a bidirectional periodicity bridge between coverings of \mathbbZ and finite checks over \mathbbZ/N\mathbbZ suitable for consuming future SAT‑style search output. All 63 published theorems depend on exactly propext, Classical.choice, and Quot.sound: no sorry, no native_decide, no solver in the trusted base. The mathematical content is known ‑‑ the density argument is folklore, and far larger uncertified classifications of covering numbers exist ‑‑ so the contribution is epistemic rather than mathematical: these exclusions are theorems of the Lean kernel, with an axiom gate enforced mechanically in continuous integration.
Authors:Jielun Peng, Yabin Wang, Yaqi Li, Jincheng Liu, Xiaopeng Hong, Athanasios V. Vasilakos
Abstract:
Generative AI has rapidly expanded audio‑visual forgery beyond human‑centric deepfakes into general scenes. Existing AIGC detection methods assume audio‑visual content correspondence, identifying forgeries by spotting cross‑modal inconsistencies. However, we empirically find that this assumption does not consistently hold in general scenarios. We argue that, for general audio‑visual AIGC detection, decision‑level fusion is a more robust alternative to feature‑level fusion. Therefore, we propose DAV‑Det, a decoupled audio‑visual AIGC detection system that independently models forensic evidence from each modality. The visual detector leverages multi‑granularity representations at global, patch, and segment levels to capture spatial forgery cues, while the audio detector exploits both temporal and spectral irregularities via a gated temporal‑spectral dual‑branch architecture to model acoustic artifacts. Our method ranks 1st in the General AIGC Audio‑Video Detection Challenge of the IJCAI‑ECAI 2026 DDL 2.0 Workshop, with a final score of 0.8460. Code is available at https://github.com/tuffy‑studio/DAV‑Det.
Authors:Liyun Yan, Jianming Ma, Yang Zhang, Shengcheng Fu, Zhanxiang Cao, Keqi Zhu, Yizhi Chen, Yue Gao
Abstract:
Variational Autoencoders are widely used to encode high‑dimensional and noisy observations in robotics. However, their stochastic latent creates a mismatch with Proximal Policy Optimization (PPO): an effective policy marginalizes over the latent distribution, whereas former implementations estimate its probability ratio and KL divergence using only one latent sample. We identify a fundamental but overlooked theoretical cause: naive single‑sample approximations in stochastic latent space induce significant variance and bias in the surrogate loss. To address this, we introduce P^3 (Probabilistic Policy Propagation), a distribution‑aware optimization framework for VAE‑based policies. P^3 couples moment‑based probabilistic method for stable and efficient learning with sampling‑based calibration for robust policy behavior under latent uncertainty. In our experiments, P^3 boosts data efficiency from 64.6% to >96%, reduces convergence steps by >20%. Furthermore, P^3 is evaluated on challenging humanoid parkour tasks and shows an effective foundation for VAE‑based PPO. Code is available at https://github.com/ylyem9x/P3_Open.
Authors:Robert Geirhos, Yuxuan Li, Thaddäus Wiedemer, Neha Kalibhat, Zi Wang, Mani Malek, Oyvind Tafjord, Kevin Swersky, Been Kim, Priyank Jaini
Abstract:
In the age of foundation models, a model is only as good as its prompt. For this reason, prompt engineering has become an essential technique for improving language model performance. Since video models are currently becoming foundation models for visual tasks (e.g., visual reasoning), we here ask whether they similarly benefit from visual prompt engineering: automatically modifying the task image to improve model performance. For example, for a visual physics reasoning task ("Where does the ball land, after passing a set of obstacles?"), an abstract sketch‑like scene can be turned into a photorealistic version with a simple call to an image editing model. We find that visual prompt engineering, or VIPE for short, improves video reasoning performance across tasks. In fact, for video models, visual prompt engineering can be even more effective than classic text‑based prompt engineering or test‑time scaling. Ultimately, just as text‑based prompt engineering systematically improves language model performance, visual prompt engineering can serve as a simple, compute‑efficient approach to elicit better visual reasoning performance from video models. Example videos on our project page at https://visual‑prompt‑engineering.github.io/.
Authors:Haochen Jiang, Jialei Pan, Yuzhe Sun, Zhe Dong, Lecheng Ren, Yanfeng Gu, Tianzhu Liu
Abstract:
Unmanned aerial vehicle (UAV)‑satellite cross‑view geo‑localization matches UAV images against satellite imagery and has achieved impressive accuracy on clean (non‑degraded) image benchmarks. In real‑world flights, however, UAV observations are frequently affected by adverse weather, illumination changes, platform motion, sensor noise, and compression, while the robustness of existing methods under such degradations remains largely unexamined. In this paper, we present UAVSat‑Deg, a large‑scale robustness benchmark for degraded UAV‑satellite geo‑localization, comprising University‑1652‑Deg and SUES‑200‑Deg. UAVSat‑Deg covers 27 corruption types, including 19 core and 8 compound corruptions, at three severity levels, supports bidirectional drone‑to‑satellite and satellite‑to‑drone retrieval as well as multi‑height UAV acquisition, and contains more than 11.7 million pre‑generated corrupted test images. Benchmarking representative methods under this protocol reveals substantial robustness gaps, particularly under severe and compound corruptions. To address this problem, we propose ReLATE, a Reliable Evidence Learning framework with Adaptive Token Evidence Regulation, which realizes reliability‑adaptive feature fusion during descriptor construction. ReLATE estimates a structure‑smoothed reliability field over visual tokens, aggregates trustworthy local evidence, and adaptively integrates it into query‑derived representations; the regulated query representations are then combined with the CLS‑token and GeM‑pooled branches to form the final cross‑view descriptor. Across both test sets and retrieval directions, ReLATE achieves the best average corrupted‑test performance among the compared methods while maintaining competitive accuracy on clean images. The code and dataset will be available at https://github.com/JHC626/ReLATE.
Authors:Swarnadip Chatterjee, Ssharvien Kumar Sivakumar, Anirban Mukhopadhyay
Abstract:
Computational cytology on whole‑slide images is challenging because malignant cells are rare, heterogeneous, and annotated slides are scarce. Anomaly detection frameworks can be trained on normal slide‑negative patches and then applied at test time to flag abnormal patches in held‑out slides. Most unsupervised anomaly detection approaches including generative ones (GAN‑based and diffusion‑based), are tuned to organ‑level imaging and require large curated datasets. In cytology the signal is cell‑centric: rotating or flipping a single‑cell patch does not change its diagnostic class, yet standard diffusion models treat transformed views as distinct inputs, leading to transformation‑dependent reconstructions and unstable anomaly scores. We propose a D4‑equivariant diffusion framework that enforces rotation and reflection symmetry both architecturally, via a D4‑equivariant U‑Net, and at inference, via equivariant noise coupling and (optionally) frame averaging. This alignment with biological invariance yields transformation‑consistent pseudo‑healthy reconstructions and more stable anomaly ranking under symmetry. On two publicly available cytology datasets of bone marrow and peripheral blood smears, our D4‑equivariant diffusion models achieve higher AUC and retrieve more abnormal cells in the top K predictions than non‑equivariant generative baselines, a deep one‑class, and a multiple instance learning based method, while substantially reducing score variance across rotations and flips. Code is available at https://swchmida.github.io/D4diffCyto/.
Authors:Jiaqi Yang, Jiayi Li, Yihan Fu, Hongxiao Zhao, Zhan Chen, Qiuping Wu, Yuchao Yang, Bonan Yan
Abstract:
Prefill‑decode disaggregation (PD) and roofline‑based operator placement are common strategies for partitioning Large Language Model (LLM) inference across heterogeneous systems, but they are often insufficient in practice. End‑to‑end latency also depends on workload shape, runtime device contention, and persistent weight layout. We present DOPS (dynamic operator scheduling), a hardware‑aware, closed‑loop framework that jointly optimizes operator scheduling and blockwise weight layouts. DOPS constructs a stage‑aware directed acyclic graph (DAG) and integrates two components: the Bifocal scheduler for dynamic operator‑to‑device placement and the Weight Layout Arbiter (WLA) for selecting hardware‑efficient weight layouts under strict memory constraints. Across representative heterogeneous systems combining neural processing units (NPUs) and processing‑in‑memory (PIM) devices, Bifocal achieves geometric‑mean speedups of 1.20× to 2.23× over the PD baseline. WLA provides an additional geometric‑mean speedup of 1.28× to 1.33× over Bifocal/Linear. DOPS also supports systematic analysis of workload sensitivity and hardware scalability for LLM serving. The source code is available at https://github.com/YIAI‑02/TriForm, and the visualization tool is demonstrated at https://youtu.be/Ya_oMCyYno0.
Authors:Minhyeok Lee, Chiyoung Kim, Chanhoe Gu, Seongrok Kim, Sanghyuk Roy Choi, Donghwan Hwang, Donghun Ryu, Seokhyun Kim
Abstract:
Vision‑Language‑Action (VLA) models translate natural‑language commands into robot action sequences, but leading systems on the LIBERO‑Plus robustness benchmark use three‑ to seven‑billion‑parameter backbones whose memory demands can exceed embedded robotic budgets. We present CoTinyVLA, a 0.9B‑parameter action model on a Qwen3.5‑0.8B backbone that obtains that robustness by structuring supervision instead of enlarging the model. Three components target different axes of the problem: dual‑view temporal input of 16 history frames per step with textual camera and time markers; hierarchical chain‑of‑thought (CoT) distillation from a 35B teacher into an episode‑level Plan and a chunk‑level Think span over task phase, gripper state and next subaction; and paraphrase augmentation expanding 40 base commands into 800 variants. On LIBERO‑Plus, spanning 10,030 perturbed tasks across seven perturbation dimensions, CoTinyVLA reaches 90.8% on Spatial, 87.3% on Object, 86.6% on Goal and 80.7% on Long, leading the strongest 7B baseline on all four suites by 4.7, 2.8, 15.9 and 3.0 points, with every margin interval excluding zero. The gains concentrate on the hardest axes of the benchmark: across the eleven published baselines none exceeds 53.2% on Robot Initial States in any suite, whereas CoTinyVLA reaches 73.6% on Goal against 39.9% for the strongest baseline. Ablations show the three components to be separable by perturbation axis, and at a matched image budget how frames are divided between the two cameras and across time accounts for 8.6 points on its own. Closed‑loop inference peaks at 2.25 GiB of allocated GPU memory, and paired interventions show the episode Plan to be load‑bearing: replacing it with an empty or contradictory span costs 40 to 45 points of success. Structured supervision thus lets a 0.9B backbone exceed all of them. Code: https://github.com/BrainJellyPie/CoTinyVLA
Authors:Korosh Vatanparvar, Ashutosh Joshi, Maria Xenochristou, Mohammad Abuzar Hashemi, Prasad Kasu, Deepak Bansal, Daniel Lopez-Martinez, Anchal Nema, Ramya Ganesan, Will Kimbrough, Alex Woody, Yadunandana Rao, Dilek Hakkani-Tur, Wilko Schulz-Mahlendorf
Abstract:
Health AI is evolving from answering questions to agentic systems that converse with patients, reason about health records, and act on their behalf. Primary care guards against diagnostic errors and unsafe care; agents assisting in this domain warrant evaluation against the same risks. Current benchmarks focus on medical knowledge, assessed through isolated question‑answering or clinician‑facing tasks. PatientAgentBench benchmarks patient‑facing agentic healthcare; it evaluates a foundation model, wrapped in an agent with a sandbox of healthcare tools, conversing with a simulated patient. Each conversation is scored by an LLM‑as‑a‑Jury across six dimensions via over a hundred conversation‑agnostic, clinician‑grounded criteria. To validate alignment, licensed clinicians annotated shared conversations, yielding 79‑93% adjacent agreement between jury and expert raters, on par with or exceeding clinician inter‑rater agreement. We benchmarked 10 models across four families on the same 1,200 scenarios and found clinical gaps. Triage quality is the most discriminating dimension: pass rates rise from 32% for the weakest models to 88% for the strongest, with agents often acting on administrative requests without clinical screening. Clinical safety and workflow accuracy follow the same pattern: the weakest models fail often, fabricating unexecuted actions, while frontier models fail on only 1‑3% of cases, from unverified tool outputs and omitted crisis resources in an emergency. More capable models narrow these gaps but do not close them; the strongest scores only 4.25 of 5 overall. These failures surface only in sustained, tool‑using conversations against realistic patient records, confirming that static benchmarks are insufficient as healthcare agentic systems gain autonomy. We release the framework as a reproducible, clinician‑validated evaluation standard to help the field close this gap.
Authors:Anis Ur Rahman
Abstract:
Video saliency models typically apply a single fixation strategy across crowd scenes, despite systematic changes in attention with crowd density. Sparse scenes encourage tracking individuals, whereas dense scenes shift attention toward collective motion and scene‑level landmarks. We introduce DensFiLM, a density‑conditioned video saliency model that inserts a lightweight Feature‑wise Linear Modulation layer at the bottleneck of a Video Swin Transformer. A learned density embedding produces channel‑wise scale and shift parameters, allowing the decoder to reconstruct saliency from features selected for each density regime. The module adds only ~100K parameters and can use either CrowdFix density labels or the model's own density prediction. On CrowdFix, DensFiLM achieves mean NSS 1.434 and CC 0.517 over four seeds, improving over ACLNet by 14.7% and 14.9%, respectively, while predicted‑density conditioning matches oracle‑label performance. Ablations show that explicit RAFT optical flow and larger temporal and social‑force extensions provide no further improvement in this setting. In a centre‑prior‑subtraction diagnostic, density conditioning yields an NSS gain of 0.462 over the unconditioned backbone, compared with 0.124 under standard evaluation. These results show that lightweight bottleneck conditioning provides a more effective inductive bias than increasing model capacity for crowd‑video saliency. Our code is available at https://github.com/aniskhan25/crowdfix‑saliency.
Authors:Akshay Sasi
Abstract:
Language models are almost always quantized before they are deployed, and a growing line of work asks whether quantization also lowers their privacy risk. That work measures privacy almost entirely with membership inference. We think this is the wrong thing to measure for the risk that most people actually worry about, namely a model reproducing its training data word for word, and we measure that directly. Using the Pythia models and the public set of sequences each of them is known to have memorized, we track verbatim extraction across five precision levels, from full precision down to four bits, and across three model sizes, while measuring general capability (perplexity) at every point. We find two things. Quantization is a selective forgetter: verbatim memorization falls off faster than capability at every precision and every model size we tried, and this holds under two unrelated quantization algorithms and two evaluation corpora. But the selectivity is not enough to make quantization a privacy defense, which cuts against the optimistic reading of earlier membership‑inference results. At the largest model we study, four‑bit quantization still reproduces most of the memorized sequences while giving up only a few percent of capability, and the fraction of memorized data that survives quantization grows with model size. We conclude that compression should not be treated as a way to remove memorized training data, and that extraction, not membership inference, is the number practitioners should be watching. All code, sampled evaluation data, and per‑configuration results are released.
Authors:Noor Islam S. Mohammad, Uluğ Bayazıt
Abstract:
Knowledge‑intensive multimodal question answering (KI‑MMQA) sits at the intersection of three expensive primitives: long visual token sequences, dense retrieval over large external corpora, and full cross‑modal fusion. Existing systems pay all three costs uniformly per query, even though only a small fraction of visual content and retrieved knowledge is actually relevant to any given question. We introduce SKIP (Salient Knowledge‑Injected Pathways), a unified inference architecture that routes computation along sparse pathways jointly conditioned on the question, the image, and a difficulty estimate. SKIP combines question‑guided visual token pruning, region‑conditional sparse retrieval, bipartite sparse cross‑attention, and speculative knowledge verification with an adaptive budget controller that allocates compute proportional to predicted question difficulty. We derive an information‑bottleneck bound showing that the optimal visual sparsity rate scales as O(1/\sqrtN) under realistic question‑image mutual‑information assumptions, with retained accuracy guarantees. Across five KI‑MMQA benchmarks (OK‑VQA, A‑OKVQA, InfoSeek, Encyclopedic‑VQA, and ViQuAE), SKIP matches or exceeds the accuracy of strong dense baselines while using 3.4‑‑6.8× fewer FLOPs and 2.7× less end‑to‑end latency. Code available at: https://pmlrbd.github.io/skip/
Authors:Debjyoti Paul
Abstract:
Production LLM agents are increasingly assembled from a frozen model wrapped in a harness: a prompt template, a tool set, a memory/retrieval layer, a planning strategy, and a verification policy. Two 2026 systems, Meta‑Harness (Lee et al., 2026) and HyperAgents (Meta AI, 2026), show that this harness can itself be optimized or even self‑rewritten by an agentic proposer ‑‑ at the cost of either an expensive code‑search loop or unconstrained self‑modifying code, neither of which is auditable or usable with a fully black‑box model API. We take a narrower, more constrained position: treat the harness as a small, fixed, human‑legible action space and learn a policy over it online with classic sample‑efficient reinforcement learning (an ε‑greedy contextual bandit and REINFORCE), scored against a multi‑objective reward (task success, verifier score, policy compliance, cost, latency, and an unsupported‑claim penalty). We instantiate this control system with DSPy (Khattab et al., 2024) as both the context assembler and the source of the strongest non‑adaptive baseline (a DSPy BootstrapFewShot static prompt), and evaluate it across three verifiable task domains ‑‑ tool‑use workflows, code generation (HumanEval), and multi‑hop retrieval QA (HotpotQA) ‑‑ and two model providers (a local Ollama model and AWS Bedrock). We release the harness‑control‑system code, the cross‑domain verifiable task suite, the full trajectory/reward‑decomposition logs from training, and a provider‑agnostic deployment recipe for applying this to a new organization's domain and verification setup.
Authors:Debjyoti Paul
Abstract:
A growing body of 2026 work applies control theory to LLM agents: Lyapunov‑certified stability for tool‑mediated controllers (Prinos et al., "Stable Agentic Control", 2026), sample‑complexity bounds for sparse policies over massive discrete tool universes (Majumdar, "Sparse Agentic Control", 2026), and regulatory‑control decompositions of multi‑agent systems into auditable feedback loops (Nogueira and Skogestad, 2026). We do not claim to introduce control theory to LLM agents ‑‑ that ship has sailed. Our narrower claim is about what the controlled variable is. Prior work controls tool selection, inter‑agent message routing, or the agent's raw action stream. We instead treat context assembly itself ‑‑ which prompt template, which few‑shot demonstrations, how much retrieved context, how many planning/verification passes ‑‑ as the controlled variable, learned online by a contextual bandit or REINFORCE policy sitting outside a frozen model. This paper develops the formal decomposition (inner frozen policy π_θ, outer context policy π_ϕ), gives a stability argument for the online controller in the sense used by Zhang et al. (2026) (non‑decreasing expected reward under bounded policy change), and reports an uncertainty‑calibration analysis of the controller's own confidence against realized task outcomes. The applied counterpart to this paper instantiates the same controller across three domains and two model providers and releases the dataset, trajectory logs, and a deployment recipe; here we focus on the formal framing and the stability/uncertainty evidence a control‑theoretic claim requires.
Authors:Liudas Panavas, Sebastian Minus, Bradley Monton, Derek Ray, Suhaas Garre, Sushant Mehta, Edwin Chen
Abstract:
Language‑model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let it govern every action that follows. Existing benchmarks rarely test this deployment pattern directly; they measure whether an agent can complete a task, not whether a long, binding policy document actually constrains its behavior over an extended tool‑use horizon. We present HANDBOOK.md, a benchmark of 65 agentic tasks modeled on how enterprise employees follow company handbooks. Each task places an agent in a self‑contained company environment, a file workspace together with mock email, chat, calendar, issue‑tracking, and commerce services exposed over the Model Context Protocol, and instructs it to carry out routine professional work governed by an expert‑written standard operating procedure of 20 to 124 pages. Tasks span five domains (finance, medical billing, insurance, logistics, and HR) and ten fictional companies. To resist memorization, every task modifies one of ten base handbooks, altering the specific rules and thresholds on which grading turns, so no two tasks share a policy. Grading is fully deterministic: each task carries a rubric of programmatic criteria (824 in total) that check both that required actions occurred and that prohibited actions did not. Under strict grading, where a trial passes only if every criterion is satisfied, the best of thirty evaluated model configurations passes 36.2% of trials, and most frontier configurations remain below 25%. Failures follow consistent patterns: agents let a plausible in‑environment request override the standing policy, perform a required check and then act against its result, lose rule details over long horizons, and report compliance they did not achieve. We release all tasks, environments, and the evaluation harness.
Authors:Yizhou Chen, Hang Xu, Dongjie Yu, Yupu Lu, Tengye Xu, Zeqing Zhang, Wei Zhang, Yi Ren, Ben M. Chen, Jia Pan
Abstract:
Successfully automating dexterous, long‑horizon robotic manipulation requires frameworks capable of both high‑level reasoning and fine‑grained execution. Traditional task and motion planning (TAMP), while excellent at symbolic planning, is often brittle in contact‑rich operations. Simultaneously, imitation learning (IL), while effective in manipulation tasks with visual feedback, is limited by its low capability in spatial generalization and multi‑stage operation. To reconcile their complementary strengths and limitations, we propose DR‑LfD (Decomposed and Reorganized Skills Learned from Demonstrations), a framework that seamlessly integrates visuomotor policies into a TAMP‑gated decision‑making system. Based on contact relationships, DR‑LfD decomposes human demonstrations into atomic skills, which are reproduced as visuomotor policies or object‑centric primitives. The initiation, termination, and constraints of the visuomotor policies are carefully modeled and implemented in a TAMP‑compatible form, enabling reorganization of skills learned from different sources. DR‑LfD transforms the learning problem from one requiring exponential demonstration data over possible skill sequences to one whose demonstration burden scales with the number of distinct skill types, with limited data for each skill. Through comprehensive real‑world and simulation benchmarking across diverse scenarios, we demonstrate the strong performance of DR‑LfD on tasks involving multiple steps, unseen setups, and physical constraints. Project website: https://dr‑lfd.github.io/DR‑LfD‑website.
Authors:Siyuan Xu, Yan Wang, Haofei Song, Lili Gao, Jiansheng Wang, Qing Zhang, Dan Huang, Boxiang Yun, Hongkai Xiong, Qingli Li
Abstract:
Histopathological examination primarily relies on hematoxylin and eosin (H&E) and immunohistochemistry (IHC) staining. Although IHC provides critical molecular information, it is costly and requires specialized expertise. Stain transfer provides an efficient alternative by computationally generating IHC from H&E images, but remains challenged by unified and interpretable modeling for heterogeneous biomarkers under pixel‑unaligned supervision. We propose DMCoStain, a novel Data‑Model Co‑optimization framework for Stain transfer. It iteratively co‑refines training data and model capability, improving staining accuracy and interpretability in both pathological and structural consistency. To refine training data in a clinically meaningful manner, it incorporates the Multimodal Expert‑Guided Finer Selection (MEGFS) strategy, built upon a pioneering IHC‑positive‑expression (IPE) vision‑language model (VLM) that emulates pathologist reasoning. To support MEGFS, we construct ImmunoInstruction, the first large‑scale IPE instruction‑following dataset with 150K VQA samples. Extensive experiments on multiple tissues and biomarkers demonstrate that DMCoStain achieves state‑of‑the‑art (SOTA) accuracy. This paradigm offers strong practical value, and MEGFS also functions as a specialized evaluation tool for future model development. Dataset, code, and more details are in https://github.com/SikangSHU/DMCoStain.
Authors:Tianyu Li, Jiahao He, Keren Fu, Qijun Zhao
Abstract:
We introduce RDVSv2, a large‑scale benchmark for RGB‑D video salient object detection (RGB‑D VSOD) with dense frame‑level annotations. Existing datasets in this emerging field are often limited in scale and annotation quality, while also relying on less geometry‑consistent depth cues. To address these limitations, RDVSv2 is built from publicly accessible stereoscopic online videos and contains 249 video sequences with 29,077 annotated frames. It includes depth maps derived from stereoscopic videos, together with frame‑wise salient object masks annotated with eye‑tracking guidance. Compared with existing datasets, RDVSv2 is much larger in scale and covers more diverse and challenging scenarios. In addition, we establish a strong baseline for RGB‑D VSOD based on Segment Anything Model 2 (SAM2). Specifically, we employ a parameter‑efficient fine‑tuning (PEFT) strategy to adapt the SAM2 encoder to jointly encode RGB, depth, and optical flow cues. Extensive experiments show that RDVSv2 is substantially more challenging for existing RGB‑D VSOD methods. Meanwhile, the proposed baseline achieves state‑of‑the‑art results on RDVSv2 and existing RGB‑D VSOD benchmarks. We hope that RDVSv2 and the provided baseline will serve as useful resources for future research on RGB‑D VSOD and related multi‑modal video understanding tasks. Our dataset and code will be available at https://github.com/ltynick/RDVSv2.
Authors:Jaeha Kim, Kyoung Mu Lee
Abstract:
Degraded images not only reduce visual quality but also impair downstream high‑level vision tasks. Task‑driven image restoration (TDIR) addresses this issue by jointly optimizing restoration quality and task performance. Recent works show that pretrained diffusion priors benefit TDIR, yet diffusion‑based restoration is inherently stochastic, as the sampling process depends on a random noise term, which can undermine task consistency. In this paper, we show that a deterministic, noise‑free one‑step forward pass with pretrained diffusion priors can substantially improve TDIR, but the benefit critically depends on the adaptation module: LoRA yields consistent gains, whereas ControlNet‑style conditioning does not. This enables one‑step forwarding that surpasses conventional multi‑step diffusion TDIR baselines. Furthermore, we introduce a task‑preserving GAN training strategy that improves perceptual quality without sacrificing task performance. Extensive experiments on classification, segmentation, and detection demonstrate consistent gains over prior TDIR methods, and we further validate generalization on real‑world degraded images and OCR.
Authors:Zhouheng Li, Fangguo Zhao, Mattia Piccinini, Baha Zarrouki, Yuan Gao, Zitong Shan, Johannes Betz, Chen Lv, Lei Xie
Abstract:
Autonomous multi‑vehicle racing requires real‑time planning of diverse competitive behaviors in intense interactions. Existing planners often struggle to balance strategic diversity and computational efficiency. To address this challenge, we propose Sampling‑based Game‑Theoretic Planning (SGTP), a real‑time framework that combines game‑theoretic reasoning with GPU‑accelerated sampling of control sequences and dynamics rollouts. Sampled trajectories are ranked using a game‑aware cost to capture competitive interactions and generate diverse racing behaviors. Our planner then performs feasibility selection by explicitly enforcing track‑boundary and dynamic collision‑avoidance constraints, ensuring safe and reliable transitions between racing strategies. Extensive simulations on challenging tracks show that SGTP achieves a 95.24% win rate and a 99.35% task‑completion ratio in highly interactive races, with a mean computational time of 0.095 s over multiple iterative solving steps. We also demonstrate the successful application of SGTP in large‑scale scenarios with up to 10 agents. We release our code and provide an open‑source benchmark of multi‑agent autonomous racing algorithms to facilitate future research. Project page: https://sgtp‑racing.github.io/.
Authors:Huwei Ji, Jiajie Su, Yuyuan Li, Xiaohua Feng, Chaochao Chen
Abstract:
LLM‑based Cross‑Domain Sequential Recommendation (CDSR) leverages LLMs to enhance target performance via deep semantic reasoning, alleviating the dependency on overlapping users. Among LLM‑based paradigms, model merging is particularly promising for multi‑domain scenarios due to its superior scalability and flexibility in integrating diverse knowledge sources. However, our empirical investigations reveal two critical bottlenecks: (1) cross‑domain knowledge conflict; and (2) performance saturation in multi‑domain fusion. Our analysis attributes these phenomena to parameter‑level misalignment and statistical homogenization during the merging process. To address these bottlenecks, we propose SharpRec, Sharpness‑aware Model Merging with Salience Recovery for LLM‑based CDSR, a framework designed to lift the performance upper bound of merged models. SharpRec incorporates two synergistic modules: Sharpness‑aware Geometric Alignment to establish a stable geometric foundation for interference‑free fusion; and Preference Salience Activation to effectively recover the distinctive features essential for bolstering target domain performance. Extensive experiments in both dual‑domain and multi‑domain scenarios demonstrate that SharpRec consistently outperforms state‑of‑the‑art baselines.
Authors:Yunfan Bai, Yuwen Qian, Cheng Zeng, Zhen Mei, Zhaohui Yang, Wei Zhu, Shuning Zhang, Feng Shu
Abstract:
Semantic integrated sensing and communication (ISAC) is envisioned as a promising paradigm for efficient and intelligent connectivity in future wireless networks. However, the open wireless channel exposes the dual‑functional waveform to detection, which challenges the joint guarantee of covertness, sensing fidelity, and semantic accuracy. To address the challenge, we propose CoSMIC, a novel covertness‑oriented semantic ISAC framework, where the sensing output is embedded into a dual‑functional ISAC waveform through semantic modulation. Specifically, a semantic rotation coding scheme is established to map semantic latents onto the pairwise rotation and scaling of Gaussian reference sequences, which satisfies a derived closed‑form covertness constraint by a differentiable budget projection. Moreover, the radar performance is analyzed to confirm an invariant matched‑filter mainlobe response and a bounded output signal‑to‑interference‑plus‑noise ratio (SINR) under the semantic embedding. Subsequently, a reliability‑guided rectified flow (RFlow) refiner is designed to effectively reconstruct high‑fidelity semantic representations from coarse observations. Simulation results demonstrate that CoSMIC improves the semantic reconstruction quality by 18% over diffusion‑based baseline schemes with substantially reduced inference latency under strict covertness constraints, which validates the applicability to practical ISAC scenarios. The source code and video demonstrations are available at https://github.com/LanceAnlan/CoSMIC‑covertness‑oriented‑semantic‑ISAC‑framework.
Authors:Gyeongmin Kim
Abstract:
Some text‑to‑speech systems ship a synthesis model and preset style vectors but not the reference encoder that turns audio into such a vector. The model still accepts a style vector; a user with a voice of their own cannot produce one. We solve for that input directly, inverting the released pipeline by gradient descent: every weight stays frozen and only the style vector is optimized, against time‑pooled WavLM statistics of one recording. Because the objective discards the time axis, the synthesized text may differ from the recording, so no transcript and no alignment are needed. On 154 speakers from two corpora, ECAPA‑TDNN similarity rises from 0.132 to 0.413 and ResNet from 0.099 to 0.401, improving for every speaker; a verifier at its equal‑error point accepts 53% of the recovered voices as the target, against 1% for the presets they start from.
Authors:Gyeongmin Kim
Abstract:
This is an implementation and measurement study of what it costs to run a streaming speech enhancer on a CPU. We port FastEnhancer‑Medium at 48 kHz to faster‑enhancer.c, a C runtime with six int8 GEMM tiers selected at initialization, leaving architecture and weights untouched. One Apple M2 core reaches 0.069 real‑time factor, against 0.230 for the fp32 ONNX Runtime graph on the same machine, a 3.3x speedup. A Galaxy S23+ (Snapdragon 8 Gen 2) reaches 0.096. The speedup comes from specializing every layer of the runtime around one fixed model. Activation ranges are recomputed per frame, so no calibration set is needed; the k=3 convolutions use Winograd F(2,3); cross‑stage state is fp16; the GRU and the dequantization epilogues are fused; and nothing is allocated after startup. Over 824 VoiceBank‑DEMAND utterances the engine tracks fp32 to within ‑0.006 PESQ and ‑0.08 dB SNR. Speed alone does not settle deployment cost. The enhancer holds a fraction of a core for as long as the microphone is open, so its real‑time factor is a duty cycle. A benchmark races through a file; an audio callback does not. Pacing to the 6.67 ms deadline costs 4.2x per frame, saves 49% of the energy, and leaves the cheapest core placement missing 96% of its deadlines. All SIMD tiers within an architecture family emit byte‑identical output. The runtime is released as a dependency‑free library.
Authors:Jiaxin Bai, Jiaxuan Xiong
Abstract:
Joint‑Embedding Predictive Architectures (JEPAs) learn world models by predicting in representation space rather than reconstructing pixels, making them a natural backbone for latent model predictive control from offline demonstration logs. JEPA‑style training optimizes short‑horizon latent prediction, whereas planning requires a multi‑step ranking of imagined futures by goal progress. Prior JEPA planners often inherit that ranking from embedding geometry, typically latent Euclidean distance, which arises as a byproduct of representation learning rather than as a progress cost mined from the logs. We propose temporal‑distance JEPA (TD‑JEPA), which retains the LeWM encoder‑‑predictor backbone and mines a directed temporal cost from reward‑free trajectories: same‑trajectory step order supplies positive targets, cross‑trajectory pairs act as heuristic negatives, and a rollout‑consistency term matches the planner horizon. The mined supervision serves two roles: as the deployed planning cost when progress is topological, and as a representation signal that improves Euclidean planning when contact geometry dominates. Under locked evaluation, deploying the mined cost raises Two‑Room success to 100.0% versus LeWM's 97.4%, while shared Euclidean planning on the same temporally trained checkpoint raises OGB‑Cube by 14.2 points over LeWM and improves Push‑T. Against LeWM and the concurrent RC‑aux baseline under locked evaluation, TD‑JEPA matches or exceeds both methods on every environment. Ablations show that the directed head, cross‑trajectory negatives, and rollout consistency each contribute. TD‑JEPA narrows the train‑‑plan gap for JEPA world‑model planners by discovering temporal progress structure in offline logs and co‑designing cost form with plan‑time deployment. Code is available at https://github.com/HKBU‑KnowComp/TD‑JEPA.
Authors:Qian Cheng, Saad Mohammad Rafid Pial, Ruize Tang, Yiming Su, Emilie Ma, Finn Hackett, Ivan Beschastnikh, Yu Huang, Tianyin Xu
Abstract:
Specula is a push‑button agentic system that generates high‑quality formal specifications for large, complex system code and uses the specifications for highly effective model checking and bug finding. Specula employs large language model (LLM) based coding agents to autonomously develop TLA+ specifications, including invariants that describe correctness properties of the target system and formal models that describe the system implementation with the right level of abstractions. Specula is fully autonomous and thus eliminates the barrier of applying formal methods to real‑world system code (as in traditional human‑centric approaches). Meanwhile, Specula addresses limitations of LLM‑driven techniques like reward hacking and hallucinations through self‑evolving loops that iteratively improve specification quality by enabling the agents to deepen their understanding of system code and its behaviors. We have used Specula to check 48 open‑source system projects; Specula found 249 bugs including many deep bugs that are hard to find by existing approaches. Specula has been used by several companies and is maintained at https://github.com/specula‑org/Specula.
Authors:Sagar Lekhak, Prasanna Reddy Pulakurthi, Emmett J. Ientilucci
Abstract:
Hyperspectral imaging (HSI) is useful for material discrimination, but operational mine screening also depends on how many false alarms must be inspected before targets are found. This paper studies PFM‑1 landmine detection in unmanned aerial vehicle (UAV) visible and near‑infrared (VNIR) HSI using spectral angle mapper (SAM), matched filter (MF), adaptive coherence estimator (ACE), and constrained energy minimization (CEM). We compare a ground‑measured SVC signature, a fully informed in‑scene core‑pixel signature, and a simulated human‑in‑the‑loop signature bootstrap. Besides receiver operating characteristic area under the curve and average precision, we report target‑discovery curves and spatial candidate‑review counts. Full‑review bootstrapping reaches the fully informed in‑scene signature case after all seven target regions are verified, but the required inspection effort varies strongly: ACE confirms all regions in two rounds and nine candidate inspections, whereas the SAM variants need thousands of candidate reviews for their final target locations. Code is available at https://github.com/SagarLekhak/IEEE_WHISPERS_2026_UAV_HSI_PFM1.
Authors:Yu Wang, Yi-Kai Zhang, Wentao Shi, Ziang Ye, Yuchun Miao, Yueqing Sun, Qi Gu, Xunliang Cai, Lan-Zhe Guo, Han-Jia Ye, Fuli Feng
Abstract:
Training large language models (LLMs) to act in long‑horizon games is a promising step toward generalist decision‑making, yet reinforcement learning with verifiable rewards (RLVR) relies on sparse final rewards that reveal little about which decisions determine success. Denser process signals could supply this missing turn‑level credit, but existing sources are hard to keep both cheap and accurate. We observe that changes in a game solver's state value reveal whether an action advances the state toward success. Building on this insight, we propose CAST (Credit Assignment from Solver Teachers), which converts these value changes into solver advantages and injects them into RLVR as turn‑level signals. We further show that, under a soft‑optimal solver assumption, maximizing the solver advantage is equivalent to on‑policy distillation from the solver, requiring only scalar values rather than teacher logits. Across Sokoban, Minesweeper, and Rush Hour, CAST outperforms all trained baselines on every game under both in‑domain and unseen‑difficulty evaluation and achieves the highest average zero‑shot performance on ALFWorld and WebShop. Our code is available at https://github.com/Wloner0809/CAST.
Authors:Lai Wei, Chengqi Li, Jiapeng Li, Ruina Hu, Yue Wang, Weiran Huang
Abstract:
Real‑world tasks often require models to learn from task‑specific context rather than relying only on pre‑trained knowledge. While recent work has highlighted this capability as context learning, existing evaluations mainly focus on textual contexts. In many practical settings, however, the context to be learned from is multimodal: scientific findings are conveyed through figures and tables, financial indicators are scattered across converted reports, and spatial decisions depend on maps, scenes, or web pages. We introduce CLBench‑V, a benchmark for multimodal context learning that addresses the difficulty of localizing where context use breaks down by organizing tasks around three dimensions: context grounding, new information application, and new knowledge learning. CLBench‑V combines converted public benchmarks with newly constructed datasets spanning domains such as science, finance, long‑document understanding, spatial reasoning, and web‑based visual question answering. To reduce the cost of constructing domain‑specific context‑learning tasks, we further use automated construction and filtering procedures for our newly built datasets. Across 3,443 instances and six recent multimodal models, the best overall score is only 0.2847, indicating that multimodal context learning remains far from saturated. Moreover, InternVL3.5‑30B‑A3B performs best on context grounding and new knowledge learning, while Qwen3.5‑Plus performs best on new information application. We further analyze judge reliability, context length, image count, and representative failure cases. Code is available at https://github.com/IamLihua/CLBench‑V.
Authors:Jingbo Zhang, Haoxiang Sun, Wenbo Wang, Wenbo Zhang
Abstract:
This paper presents ContractHIL‑HLS, a contract‑aligned multi‑agent workflow for practical high‑level synthesis (HLS) engineering. The workflow makes three contributions. First, it introduces a structured contract as the semantic‑alignment and task‑execution artifact that translates natural language requirements into explicit interfaces, constraints, validation checks, and rollback rules. Second, it incorporates hardware information into the feedback loop by feeding HLS, Vivado, PYNQ runtime, power, and failure evidence back into generation, thereby extending LLM‑assisted HLS from kernel code toward system‑ and board‑level closure. Third, it decomposes agents by semantic lowering and execution tasks rather than by conversational roles: a Contract Agent lowers natural language into the contract, an HTML Agent renders the contract as persistent structured HTML, and a Hardware‑in‑the‑Loop Agent implements and revises the design with measured evidence. We evaluate ContractHIL‑HLS in two parts. On 94 locally executable HLS‑Eval tasks, the structured contract provides the largest small design gain, improving the estimated single‑sample testbench pass rate from 64.0% to 70.2%; the full flow reaches 70.4% pass@1 and 76.6% pass@5. Because HLS‑Eval does not exercise board‑level design, we also validate ContractHIL‑HLS on a board tested ML‑KEM/ML‑DSA post‑quantum cryptography (PQC) secure‑message accelerator, where the retained dual‑bitstream organization reduces six‑message average text runtime from 207.3 ms to 52.4 ms with positive routed WNS on both images while preserving decrypted‑message verification. We open‑source our work at BJUT‑CS316‑LAB/ContractHIL‑HLS (https://github.com/BJUT‑CS316‑LAB/ContractHIL‑HLS).
Authors:Zhenning Shi, Chen Xu, Junhao Zhang, Kefei Zhang, Linjie Liu, Zhedong Zheng, Tao Li
Abstract:
Real‑world Image Restoration (Real‑IR) aims to recover high‑quality (HQ) images from complex and unknown degradations. Although recent diffusion‑based methods have substantially improved perceptual quality, their current designs leave two key challenges unresolved. Methods that start from Gaussian noise are slow and often less faithful to the degraded input. Residual‑based methods usually train from scratch, which makes it hard to exploit modern pre‑trained generative priors. In this paper, we present ScaleResfusion, a scalable diffusion framework for real‑world image restoration built on pre‑trained text‑to‑image rectified‑flow models. The core of our method is Residual Rectified Flow, which introduces the residual term R into Standard Rectified Flow. Instead of starting from pure noise, it uses a residual transport path that starts from noisy low‑quality (LQ) images and admits an exact acceleration point. By learning the residual vector field, Residual Rectified Flow keeps the output distribution and linear diffusion process consistent with the pre‑trained rectified‑flow models. This makes parameter‑efficient fine‑tuning possible at scale. We further introduce a knowledge‑distillation pipeline to reduce sampling cost while maintaining restoration quality. Extensive experiments on multiple real‑world restoration tasks show that ScaleResfusion achieves state‑of‑the‑art performance with much higher efficiency. These results suggest a practical and scalable way to adapt large pre‑trained diffusion models to real‑world image restoration. Our code and models are available at https://github.com/YukinoshitaLove/ScaleResfusion.
Authors:Jiaxin Bai, Jiaxuan Xiong
Abstract:
Different research lines use the term world model in different ways, yet they share a common aim: to capture how the world evolves under action in a form that supports perception, simulation, and planning. Two prominent realizations are neural predictors that learn dynamics in continuous vector spaces, and hand‑built physics engines that expose explicit state and physical laws. Neural predictors scale from data but leave the form of the dynamics implicit; physics engines are inspectable and editable but difficult to construct at scale. We introduce VisualPatchWorld (VPW), which represents world dynamics as code. VPW first selects a qualitative dynamical form with short active probes, then fits that form's free parameters from recorded state‑action traces by minimizing multi‑step prediction error. The resulting programs can be rolled forward like a simulator, inspected in source form, and used inside model‑predictive control; image‑derived scene graphs can supply the live state at replan time. Across comparisons with prior code‑based world models, VPW attains 69.0% mean planning success and exceeds the strongest code baseline by 23.5 points. The largest gains arise when choosing the correct qualitative dynamics is essential. Under the same planner, the induced models approach ground‑truth engine success on navigation and grasp‑rich control; a residual gap remains for contact‑rich pushing, and checking a shortlist of promising plans in the engine closes most of that gap. These results establish a practical route toward automatically constructed code world models that are useful for planning. Code is available at https://github.com/HKBU‑KnowComp/VisualPatchWorld/.
Authors:Yuhang Yang, Kai Tang, Chao Ye, Haobo Wang, Qiqi Luo, Jinguang Zheng, Zhixin Zhang
Abstract:
Debt collection is a critical negotiation task in the financial industry, with strong practical relevance and exceptional academic value as a behaviorally rich, high‑stakes testbed for human‑centered dialogue systems. While large language models (LLMs) have shown promise in dialogue and negotiation, effectively evaluating their performance in this complex scenarios remains a major challenge: existing benchmarks uniformly assume users to be static, rational agents with fixed preferences, failing to capture the rich behavioral heterogeneity inherent in real‑world debt collection. To bridge this gap, we propose DebtBench, the first public persona‑enriched debt collection benchmark, that highlights behavioral heterogeneity in negotiation. Moreover, we develop DebtGPT, a debt collection agent trained to jointly optimize financial recovery and interaction experience. Our experimental results, using 16 state‑of‑the‑art LLMs, find that most existing models struggle in this complex but realistic scenarios, whereas DebtGPT outperforms all open‑source baselines and achieves performance on par with GPT‑4o. The code and data are available at https://github.com/YYuHhhh/DebtNegotiation.
Authors:Ethan Fahnestock, Erick Fuentes, Philip R Osteen, Nicholas Roy
Abstract:
We want robots to localize in previously untraversed environments against commonly available prior data. Rich semantic data available from OpenStreetMap can be useful in this task. However, existing methods either ignore this semantic information, directly matching panoramas and overhead imagery, or dramatically compress the semantic information, working with a small set of fixed classes. To leverage this rich semantic information, two challenges need to be overcome. First, useful semantic information needs to be extracted from the robot's egocentric observations. Second, the observed information must be quickly associated with the large prior semantic map (e.g., up to 628 km^2). We show that VLMs are effective at both extracting relevant landmarks from panoramas, and identifying feasible correspondences between these landmarks and prior overhead landmarks. However, using VLMs to propose all correspondences scales poorly as the number of mapped landmarks increases. Instead, we propose distilling a lightweight matcher from a VLM which computes correspondences for all entities in a map. We use this output to form an observation likelihood which is fused over time with a Bayes filter to create a time series of pose estimates. To support further investigation into generalizable cross‑view methods that leverage semantic information, we release a dataset of extracted semantics and evaluation trajectories spanning eleven environments, including panoramas we collected in a snowstorm and at night in Boston. We demonstrate our method, trained on a single city's fair‑weather data, generalizes across location, lighting, weather, and other challenges. Code and datasets are available at https://efahnestock.github.io/loci/.
Authors:Kai Li, Yupeng Deng, Ligao Deng, Zhihao Xi, Chenhao Wang, Jierui Zhang, Yingrui Ji, Yu Meng, Xiangyu Zhao
Abstract:
Oblique‑view urban remote sensing imagery inevitably exhibits geometric projection displacements between building roofs and footprints, leading to significant distortions in spatial structure. Existing approaches either ignore these deformations or handle them implicitly within segmentation‑based frameworks, where progress is dominated by general segmentation advances rather than improvements in geometric correction. In this work, we explicitly define roof‑to‑footprint offset vector (RFOV) extraction as an independent learning task that decouples geometric alignment from semantic segmentation. To support this task, we introduce the Oblique City dataset (ObliCity), the first large‑scale benchmark that integrates high‑resolution UAV imagery and globally distributed satellite data, covering diverse city morphologies and camera perspectives. Methodologically, we reformulate DragOSM into DragRoof, an ODE‑based framework inspired by human annotation behavior. By simulating the continuous process of dragging roofs toward their footprints, DragRoof learns deterministic, geometry‑consistent offset fields and adaptively determines convergence through an end token. Extensive experiments on ObliCity demonstrate that DragRoof achieves state‑of‑the‑art RFOV extraction performance, requiring fewer inference steps while delivering superior directional and length accuracy. Our dataset and model establish a principled foundation for studying projection displacement correction in oblique remote sensing imagery. The source code and dataset will be avaliable at https://github.com/likaiucas/DragRoof.
Authors:Liexin Cheng, Xue Cheng, Shuaiqiang Liu, Cornelis W. Oosterlee
Abstract:
Automated code generation is becoming an important tool in quantitative finance, where large language models can generate option pricing implementations directly from mathematical model specifications. Validating such implementations, however, requires considerably more than conventional software testing: numerical pricing methods must remain mathematically consistent, numerically stable, and reliable across a wide range of model parameters. We introduce RIDGE, an autonomous validation framework in which generated pricing implementations are subjected to structured no‑arbitrage tests, stress tests, benchmark comparisons, and consistency checks. Validation evidence is interpreted diagnostically, while the resulting knowledge is accumulated in a repository and reused across models and successive validation iterations. This enables systematic refinement of both the pricing implementation and the validation methodology. The framework is applied to five stochastic volatility models. Across these studies, all detected implementation defects are removed and, in two cases, the validation process itself leads to new semi‑analytic pricing methodologies. The supplementary material is available in the GitHub repository: https://github.com/ShQiangLiu/ridge.
Authors:Leon D. da Silva, Marcelo P. Santos, José D. da Silva, Gilson Ferreira
Abstract:
We establish a universal block‑diagonalization framework for Discrete Exterior Calculus (DEC) operators on symmetric meshes, enabling embarrassingly parallel solvers with provable FLOP reductions. We prove that the two fundamental DEC operators, the discrete exterior derivative d and the Hodge star \star, are equivariant under isometric finite group actions on simplicial complexes. The proof exploits the permutation representation induced on cochain spaces by the group action. As a consequence, any operator assembled from d and \star (including the Hodge Laplacian, the codifferential, Maxwell‑type operators, and elasticity operators) inherits a block‑diagonal structure in a single symmetry‑adapted basis, which is computed only once per mesh. Unlike spectral methods restricted to flat Platonic domains, the framework applies natively to curved manifolds and is applicable in principle to computational electromagnetism and geometric fluid simulation on symmetric domains. Numerical experiments on a geodesic sphere (I_h symmetry) and a hexagonal torus (D_6h symmetry) yield FLOP‑based parallel speedups, relative to a dense direct factorization, of up to 62× and 182×, respectively. A further experiment on a body‑centred‑cubic (BCC) tessellation of the flat 3‑torus T^3 with T_d symmetry confirms equivariance of the exterior derivative, Hodge star, and Hodge Laplacian at machine precision for form degrees k=0,1,2 across three mesh resolutions. The FLOP‑based sequential speedup approaches its theoretical asymptote of \approx 9.07×, which a standard Schur‑multiplicity reduction deepens by a further factor of order |G|. These results show that a single symmetry‑adapted basis reduces the linear‑solve cost of structure‑preserving DEC computations on curved and three‑dimensional meshes.
Authors:Adarsh Singh, Kushal Raj Bhandari, Jianxi Gao, Soham Dan, Vivek Gupta
Abstract:
The ability to retrieve relevant tables for answering questions is a key task for structured information retrieval. Multi‑stage retrieval systems rely heavily on rerankers to refine candidate lists produced by efficient first‑stage retrievers. As a result, neural rerankers and LLM‑based reranking methods have become increasingly important due to their superior capacity for semantic understanding and reasoning compared to conventional sparse or dense retrieval models. Recently, Large Reasoning Models (LRMs) equipped with explicit chain‑of‑thought (CoT) reasoning have shown strong improvements in ranking quality in unstructured passage retrieval. In this work, we present TabRank, a framework for training reasoning rerankers for Tabular Retrieval. We first present a comprehensive dataset of 6728 reasoning traces for tabular reranking on the Natural Questions Tables dataset. We then explore two variants of training a compact reasoning model on these reasoning traces: explicit CoT distillation and conditioning the student reranker on the teacher's reasoning trace within the prompt. We stress‑test TabRank on several out‑of‑distribution generalization settings on diverse domains and multi‑table scenarios. Our approach significantly improves performance across a variety of table retrieval datasets, increasing Acc@10 by 30.5% on HybridQA, 15.2% on SQA, 52.9% on TabFact, and 13.1% on TATQA subsets of the Multi‑Table QA Benchmark compared to the base model. Notably, TabRank generalizes effectively to multi‑table reasoning. Our code, data and models are available at https://github.com/AdarshSingh7647/TabRanker
Authors:Yubo Sun, Chunyi Peng, Yukun Yan, Zhenghao Liu, Sen Mei, Bangrui Xu, Xuanhe Zhou, Chi Chen, Maosong Sun
Abstract:
Deep research requires models to retrieve, connect, and synthesize evidence from large‑scale heterogeneous sources to answer complex queries and produce analytical reports. Existing benchmarks mainly evaluate final outcomes, such as answer correctness, report quality, or citation alignment, while providing limited visibility into whether evidence is correctly selected, linked, and aggregated into supported claims and conclusions. To address this gap, we introduce HiEviDR‑Bench, a benchmark for evaluating Hierarchical Evidence Aggregation in Deep Research. HiEviDR‑Bench covers open‑domain and academic‑domain settings under both text‑only and multimodal conditions, and represents each instance with an explicit evidence graph that captures evidence selection, cross‑source linking, and aggregation from evidence to intermediate claims and final conclusions. Based on this formulation, we develop a traceability‑oriented evaluation framework with five dimensions: report quality, evidence traceability, citation accuracy, claim verification, and answer correctness, together with a progressive gating mechanism for fine‑grained error localization. HiEviDR‑Bench contains 2,000 human‑validated questions with evidence graphs across multiple difficulty levels. Experiments on 16 representative multimodal large language models show that, although many systems achieve strong report quality, their performance drops markedly on citation accuracy, claim construction, and answer correctness. Further analysis shows that the main bottlenecks lie in evidence identification and intermediate claim construction, revealing that strong surface‑level report quality does not necessarily imply grounded multi‑stage reasoning on our benchmark.
Authors:Hilaf Hasson, Aditya Chakravarty, Jayant Thomas, Krishna Gogineni
Abstract:
Recent advances in RAG aim to optimize for performance by paying high ingestion costs for knowledge ingestion: building knowledge graphs or extracting SQL tables. In this work we show that the operations that such knowledge bases allow can be replicated with zero ingestion costs (not even a vector database); in fact our solution, Zero‑Ingestion ScalableRAG, handily out‑performs all baselines (including knowledge graph approaches) in three out of the six corpora considered here, and only marginally missing maximum performance on the other three, with average accuracy across all six datasets 7.36% above the next most competitive baseline. It achieves this by keeping a workspace of document sets and values sets that it can write into and read from, allowing for on‑the‑fly aggregative reasoning in all situations where grouping is required on a primary key that is in one to one correspondence with a subset of the total document set. Capping the number of LLM calls by a constant independent of the corpus size, we also introduce Limited‑Ingestion ScalableRAG, which does use a minimal vector database as well as an automated pattern discovery from a sample of documents, to further improve accuracy at scale. Our code is available at https://github.com/cohesity/ScalableRAG .
Authors:Ce Zhang, Jinxi He, Katia Sycara, Yaqi Xie
Abstract:
Despite rapid progress in Multi‑modal Large Language Models (MLLMs), understanding long‑form videos is still bottlenecked by limited context windows. While recent keyframe sampling methods attempt to mitigate this by distilling video inputs into a compact set of query‑relevant frames, navigating the vast spatio‑temporal search space remains challenging, as spatial detail and temporal coverage often conflict. To address this, we introduce LENS, a training‑free keyframe sampling framework that dynamically decides when to zoom in for fine‑grained details and when to zoom out for broader context based on the text query. Concretely, LENS adaptively allocates a limited frame budget between spatial zoom‑ins, which highlight query‑relevant regions within individual frames, and temporal zoom‑outs, which expand the temporal scope through multi‑frame aggregation, enabling the model to reason across multiple granularities while capturing both high‑fidelity details and long‑range context. Across diverse long‑form video benchmarks, LENS consistently outperforms prior state‑of‑the‑art keyframe sampling methods and delivers substantial gains over uniform sampling, improving Video‑MME accuracy from 53.3% to 60.7% with Qwen2.5‑VL.Code is available at https://github.com/zhangce01/LENS.
Authors:Zihan Li, Feiyang Liu, Dandan Shan, Ruibo Wang, Qingqi Hong
Abstract:
Biomedical image analysis spans diverse modalities and tasks, yet real‑world deployment is hindered by severe distribution shifts across scanners, protocols, and patient populations. High‑performing models consequently require repeated domain‑specific fine‑tuning, which is a costly cycle that becomes impractical when labels are scarce or privacy constraints limit data sharing. We propose OPERA (Offline Policy‑guided Expert Routing and Adaptation), a multi‑agent ensemble framework that addresses this deployment bottleneck by treating expert weight assignment as an offline policy learning problem: a routing policy is learned from a small validation set without gradient updates to any expert agent, then deployed with test‑time adaptation to handle distribution shift. OPERA coordinates heterogeneous specialist agents through complementary mechanisms. The expert profiling module learns selection policies offline, enabling informed allocation of expertise. Each agent undergoes confidence calibration through temperature adjustment, ensuring more reliable probabilistic outputs. OPERA also incorporates distribution aware adaptation, where class weights are dynamically adjusted at the batch level using statistics derived from unlabeled test data. Instance level routing assigns each sample to the most suitable expert by leveraging inter model agreement and predictive entropy. We evaluate OPERA on 9 datasets covering fundus photography, chest X‑ray, CT, MRI, and multimodal diagnostic benchmarks, comparing against 30+ baselines across classification, segmentation, and multimodal settings. OPERA consistently improves performance and calibration quality, demonstrating that offline policy‑guided expert agents coordination is a practical path to deployable biomedical AI without retraining. Code is on \hrefhttps://github.com/HUANGLIZI/OPERAGitHub.
Authors:Jelin Raphael Akkara, Filippo Ziliotto, Luciano Serafini, Lamberto Ballan, Tommaso Campari
Abstract:
Embodied AI increasingly relies on queryable semantic maps built from pre‑trained vision‑language models to enable zero‑shot Object Goal Navigation (ObjectNav). However, existing approaches typically depend on text‑only queries, which become less reliable as semantic specificity increases toward fine‑grained object categories. We introduce IMPRINT, a zero‑shot plug‑and‑play framework that enriches textual object queries with web‑sourced images to improve grounding in queryable maps. Retrieved images are encoded using a vision‑language model, matched against the semantic map to produce similarity maps, and aggregated to yield context‑aware localization. Notably, this requires no training or modification of the underlying navigation policy. To explicitly evaluate long‑tail behavior, we present HSSD‑rare, a new ObjectNav benchmark built on Habitat Synthetic Scenes and featuring semantically specific subcategories. Across both OVON and HSSD‑rare, image‑conditioned queries consistently improve object grounding and yield end‑to‑end navigation gains. Further analysis reveals that translating localization gains to navigation performance depends critically on downstream detection quality, highlighting a key systems bottleneck in long‑tail embodied navigation.
Authors:Cesare Spinoso-Di Piano, Verna Dankers, Marius Mosbach, Jackie Chi Kit Cheung
Abstract:
Human language is driven by unspoken beliefs and belief updates, making these critical to model for successful communication between large language models (LLMs) and their users. In this paper, we evaluate the ability of LLMs to recognize unspoken beliefs made through implicatures and to understand their updates through implicature cancellation: the pragmatic phenomenon whereby an utterance's implied meaning is weakened or negated. We create the first expert‑annotated implicature cancellation dataset, [DatasetName], crowdsourced for human judgements of implicatures and their corresponding cancellations. We find that LLM belief update understanding lags behind that of humans, especially in more naturally‑occurring scenarios. Additional control experiments suggest that successes in LLM belief updates may stem in part from a reliance on prior beliefs, and that failures in belief updates may depend on their type and on their form. Overall, our study suggests that current LLMs have not yet reached human‑level understanding of unspoken beliefs and belief updates. Code and data are available at https://github.com/cesare‑spinoso/ImplicatureX.
Authors:Dengzhe Hou, Lingyu Jiang, Fangzhou Lin, Kazunori D Yamada
Abstract:
LLM cognitive scores are increasingly summarized as per‑ability profiles whose dimensions should converge across tasks, respond selectively to matched interventions, and generalize beyond the models used to define them. We introduce CogArena, a procedurally generated 13‑paradigm benchmark built around a multimethod framework for determining when cognitive‑task scores warrant dimensional labels across five theory‑motivated groupings. Across 55 open‑weight models, nearly all paradigm correlations are positive and a common axis explains about half the variance. The within‑grouping advantage is small, scoring‑sensitive, and uncertain across model families. In a separately frozen, fully crossed study across 12 models from six families, targeted scaffolds show a small matched‑grouping advantage, but no scaffold‑specific contrast survives multiplicity correction and selectivity does not improve held‑out‑family prediction. The frozen confirmation criterion fails. A post‑hoc alternate‑wording replication produces a smaller positive estimate and again fails. Together, these results support a boundary conclusion. Theory‑aligned prompting produces a small in‑battery diagonal tendency, but the present evidence does not establish stable five‑dimensional profiles. CogArena provides a workflow joining behavioral signatures, covariance, matched interventions, and out‑of‑family prediction before cognitive labels are attached to model scores.
Authors:Daniel Layeghi, Thomas Corbères, Calum Arnott, Aditya Kamireddypalli, Hashim Al-Obaidi, Steve Tonneau, Michael Mistry
Abstract:
Differentiable simulation can accelerate contact‑rich trajectory optimisation by exposing local sensitivities of task outcomes to controls. Existing approaches either use finite differences, which are expensive and step‑size sensitive; differentiate iterative contact solvers by unrolling automatic differentiation (AD), which stores a growing computation trace; or require intricate, solver‑specific KKT sensitivity derivations. We introduce an AD‑assisted implicit derivative for regularised smooth contacts and apply it to Mujoco MJX, based on the Implicit Function Theorem (IFT). The method differentiates the stationarity residual at the tolerance‑converged solution, avoiding both solver unrolling and hand‑assembled KKT systems. IFT keeps compiled temporary memory nearly constant with solver effort, changing by less than 4% from one to ten iterations versus 10.6× growth for unrolled AD. IFT memory grows slower with active contacts and model dimension, using 20× less memory at 256 contacts and 6× less at 16 contacts and 96 DoF. We further introduce optimiser distillation for residual MPC, amortising batched full‑horizon iLQR into a policy that guides short‑horizon residual iLQR. Across Finger, Franka, and Unitree, this raises six‑step success by 28‑98 percentage points over standard iLQR.
Authors:Gregorio de la Fuente, Jesse Thaler
Abstract:
Providing a practical and hadron‑level definition of multiple jet flavors has been a long‑standing challenge in collider physics. Previous work has introduced a data‑driven, operational definition of quark and gluon jets, but no robust generalization beyond two jet categories presently exists. To address this, we introduce a machine‑learning framework called "simplex demixing'' to extract T jet flavors (or topics in the statistics literature) from M data samples (or mixtures) with minimal constraints. Intuitively, our procedure identifies the maximally separable categories in the data, translating a multi‑category classifier on the M mixtures into a bounded geometric object with T vertices. We first demonstrate our procedure on a toy problem to infer the truth‑level fractions of down‑quark, up‑quark, and gluon jets from synthetic mixtures of the three pure samples. We then propose a tag‑and‑probe strategy to extract multiple light‑flavor categories in a more realistic collider setting involving dijet production. As expected, the identifiability of jet flavors depends on their relative abundance in the samples and the hadron‑level information available to the classifier architecture. Our work opens the door to data‑driven extractions of multiple jet flavor properties at the Large Hadron Collider.
Authors:Senqiao Yang, Kaichen Zhang, Zhaoyang Jia, Jinghao Guo, Yifei Shen, Xinjie Zhang, Xiaoyi Zhang, Haoqing Wang, Xiao Li, Peng Zhang, Xiang An, Yin Xie, Zhening Liu, Xun Guo, Jiahao Li, Shicheng Zheng, Jinglu Wang, Zongyu Guo, Wenxuan Xie, Zihan Zheng, Yuxuan Luo, Bin Li, Yan Lu
Abstract:
Standard vision‑language models (VLMs) suffer from Moravec's paradox: they excel at complex offline visual reasoning but struggle with simple streaming perception tasks and process them inefficiently. We present Mage‑VL, an efficient codec‑native streaming foundation model for real‑time multimodal understanding and interaction. At its core, our custom tokenizer, Mage‑ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy‑rich regions using motion vectors and residual energy across sparse anchor (I) and predicted (P) frames. Operating at a 16 x 16 patch level, this reduces visual token consumption by over 75% while preserving spatiotemporal context. Trained from scratch on approximately 560M unlabeled images and 100M unlabeled video frames, Mage‑ViT matches or outperforms flagship encoders trained on billions of image‑text pairs. We establish AI4AI data pipelines encompassing prompt‑code joint optimization for multimodal captioning and AI‑driven performance diagnosis to guide training recipes. Furthermore, through a bio‑inspired dual‑system architecture ‑ a lightweight System 1 event gate and a causal System 2 decoder ‑ Mage‑VL enables proactive streaming perception. Extensive evaluations show that Mage‑VL‑4B matches Qwen3‑VL‑4B on static tasks while achieving strong gains in video understanding and 2D/3D spatial reasoning, with up to a 3.5x wall‑clock inference speedup, and comprehensively surpasses the 15B Phi‑4‑reasoning‑vision baseline. Beyond model artifacts, we deliver seven key empirical findings covering pre‑training data efficiency, variable‑resolution scaling, codec system acceleration, VideoQA SFT redundancy, motion‑spatial synergy, AI4AI data pipelines, and Zero‑Vision SFT for multimodal RL.
Authors:Chandan Kumar Sah, Xiaoli Lian, Li Zhang
Abstract:
Repository‑level code generation relies on heterogeneous evidence whose relevance, compatibility, and completeness are inherently uncertain. Similar‑code examples, repository context, and project‑specific APIs may provide complementary information, but can also introduce noisy, redundant, or conflicting signals. Existing retrieval‑augmented approaches primarily optimize retrieval relevance without explicitly modeling how uncertainty in retrieved evidence affects downstream generation. We introduce OpenCoder, an uncertainty‑aware framework that estimates source‑specific uncertainty, uses it to filter and rank heterogeneous evidence, and guides generation, verification, and repair. A factorial analysis over API knowledge, repository context, and similar‑code evidence reveals no universal additive source ranking; instead, significant cross‑source interactions depend on the accompanying evidence and LLM backend. On an expanded 32‑task RepoExec‑inline evaluation, OpenCoder improves GPT selected‑output correctness over Baseline RAG from 56.25% to 78.13%. However, it matches a verification‑and‑repair control, and the corresponding Gemini improvement is not statistically supported, indicating backend‑dependent benefits. Target‑aware API refinement also substantially improves API‑set retrieval. These findings support treating uncertainty as an actionable control signal for repository‑level retrieval, verification, and repair.
Authors:Guiling Guo, Jia Yang, Jiahao Xu, Shuyuan Zheng, Zhonghai Sun, Qiyuan Li
Abstract:
Phenotype‑driven diagnostic benchmarks usually report the rank of the reference disease, but they rarely reveal which plausible alternatives are ranked above it or what evidence a tool‑using model examines before making its decision. We introduce GraphRareBench, a provenance‑preserving benchmark containing 2,365 ontology‑derived cases and 18,093 target‑confounder pairs. Each case includes a coarsened HPO query, a fixed candidate pool, graph‑defined hard confounders, and source‑linked evidence records. On the 237‑case gene‑component‑disjoint test split, supervised rankers using a shared 21‑feature interface achieved MRRs ranging from 0.640 to 0.740 and case‑averaged target‑over‑confounder accuracies ranging from 0.898 to 0.916. Agents instantiated with Agents‑A1 and DeepSeek‑V4‑Flash achieved MRRs of 0.746 and 0.718, respectively. Their paired MRR difference was not statistically significant, whereas their target‑evidence coverage differed by 0.561. Together with the observation that 22.1% to 43.7% of selected Hit@10 successes still ranked at least one graph‑defined hard confounder above the target, these results indicate that full‑pool retrieval, hard‑confounder discrimination, and observable evidence access capture complementary aspects of model behavior. GraphRareBench therefore provides a foundation for more transparent and evidence‑aware evaluation of phenotype‑driven diagnostic systems. Code and data are available at https://github.com/GUI0609/GraphRareBench.
Authors:Ge Zhang, Jingru Cheng, Huiyuan Chen
Abstract:
Large language models (LLMs) used as listwise rerankers in recommendation systems suffer from position bias when serializing candidate sets into prompts. We show this order sensitivity creates an exploitable attack surface: an attacker can promote a label‑0 target into the top‑k solely by reordering candidates, without changing item content, labels, or model parameters. We introduce \mathrmpromo@k to quantify this vulnerability, measuring the fraction of label‑0 targets that can be elevated into top‑k rankings via permutation. Evaluating across three domains (MovieLens, Amazon Books, and Amazon Fashion), \mathrmpromo@5 reaches up to 0.57 at an attack budget of R = 50 orderings. Furthermore, ordinary permutation stability predicts vulnerability without running the attack. While a bidirectional T5 encoder scorer reduces exposure, permutation‑consistency regularization and architectural invariance effectively mitigate it. Pointwise scoring avoids the bias issue but degrades ranking quality. These results demonstrate that input candidate order in listwise LLM reranking is a security‑relevant attack vector. Code and data are available at https://github.com/geoz‑lab/position_bias_attack.
Authors:Ke Guo, Changle Qu, Jiayaqi Cheng, Xiao Zhang, Shijun Wang, Xiaoyu Zhang, Xueliang Wang, Le Zhang, Lantao Hu, Jun Xu
Abstract:
Existing public live streaming datasets suffer from three major limitations: they provide limited access to temporally evolving multimodal live content, overlook users' cross‑domain interactions between short videos and live streams, and contain only implicit behavioral signals without explicit feedback that captures users' perceived content quality and satisfaction. These limitations prevent existing benchmarks from faithfully reflecting real‑world live streaming scenarios and hinder comprehensive research on live streaming recommendation. To address these limitations, we introduce KuaiLive‑M3, a multi‑modal, multi‑domain, and multi‑feedback dataset for live streaming recommendation, collected from Kuaishou, a leading live streaming and short video platform in China. KuaiLive‑M3 covers 21,938 users and contains 35 million live streaming interactions and 111 million short video interactions, with fine‑grained timestamps and diverse user behaviors. It further provides approximately 88 million timestamped segment‑level multi‑modal embeddings that capture the temporal evolution of live streaming content, as well as 25,403 questionnaire‑based feedback records that bridge implicit user behaviors and explicit user preferences. Based on these unique signals, we establish benchmarks for cross‑domain recommendation, live stream highlight prediction, and questionnaire‑enhanced recommendation. Extensive experiments with representative baselines demonstrate that KuaiLive‑M3 provides a challenging and realistic benchmark for future live streaming recommendation research. The results further highlight the importance of modeling temporally evolving content, transferring user preferences across domains, and bridging the gap between implicit behaviors and explicit user feedback. The dataset and benchmark code are publicly available at https://imgkkk574.github.io/KuaiLive‑M3/.
Authors:Harshini Kavuru, Dwipam Katariya, Giri Iyengar, Pranab Mohanty, Kalanand Mishra, Kalanand Mishra
Abstract:
Large language models (LLMs) have been applied to sequential recommendation by formulating it as a natural language task. Previous work has improved personalization by incorporating collaborative and sequential signals through input conditioning or LLM fine‑tuning. However, existing approaches often rely on one or more of the following: LLM fine‑tuning, additional architectural modules, representation distillation, or item‑level conditioning over long interaction histories, increasing training complexity and deployment cost. We propose REPREC, a lightweight framework that reformulates LLM‑based sequential recommendation through lightweight user representation alignment. REPREC maps a fixed‑size user embedding from a frozen sequential encoder into a small set of learned soft tokens through a lightweight MLP injector that conditions a frozen LLM, leaving both pretrained backbones unchanged while training only the injector. We conducted exhaustive experiments on multiple benchmark datasets and demonstrate that REPREC consistently outperforms LoRA while remaining compatible with different pretrained sequential encoders and LLM backbones, enabling a modular and production‑friendly recommendation pipeline without modifying either pretrained component. The gains are particularly pronounced for casual and core users across all datasets, highlighting REPREC's effectiveness in low‑data regimes. Finally, when trained on short prompt histories and evaluated with longer contexts, REPREC maintains 85‑100% of LoRA's performance while reducing per‑epoch training time by an average of 1.51X, demonstrating an effective balance between recommendation quality and computational efficiency for production deployment. The code is available at https://github.com/phdbotcode/REPREC
Authors:Seongwon Seo, Seung Hwan Cho, Young-Min Kim
Abstract:
In medical multiple‑choice question answering (MCQA), Retrieval‑Augmented Generation (RAG) can supplement the domain knowledge of language models (LMs). However, since vanilla RAG indiscriminately utilizes retrieved documents, it can degrade LM performance. To address this, we propose MedJudgeRAG. Our framework represents retrieved documents as a dynamic knowledge graph (KG) composed of entities and relations. For each option, the model judges an evidence verdict from the retrieved documents and the KG. Based on the verdict combination, the model determines a knowledge utilization strategy to reason toward the final answer. These capabilities are trained via supervised fine‑tuning using structured reasoning traces generated by a teacher LM. The training employs a weighted cross‑entropy loss that differentially weights the KG and reasoning segments. Experiments on two medical MCQA benchmarks demonstrate that MedJudgeRAG consistently outperforms both vanilla RAG and parametric baselines. Furthermore, ablation analysis reveals that the dynamic KG is more effective as graph‑conditioned supervision at training time than as an explicit output at inference time. Our code is available at https://github.com/hyu‑amllab/medjudgerag, and the generated reasoning traces are released at https://huggingface.co/datasets/youarethewon/medjudgerag.
Authors:Ferdinand M. Schessl
Abstract:
AI Engram (Kwon et al., 2026) formalizes the four engram criteria of neuroscience as a constrained inverse problem in weight space and solves it closed‑form: concept‑specific memory traces become linear objects that can be extracted once and combined arithmetically. Appendix F states the Compositional Memory States Hypothesis: edited models live on "a commutative manifold where the integration of A and B reaches a consistent equilibrium regardless of the learning sequence." The evidence base is single and paired edits ‑‑ in materials terms, single‑cycle tests, in which fatigue accumulation is structurally invisible. Whether the hypothesis holds under sequential load is exactly the "temporal dynamics" question the paper defers to future work. We run that test on the authors' own reference implementation, at their reported best edit strength (TOFU alpha=0.6, a choice favoring the linearity hypothesis), with pre‑registered predictions, across three model charges (two vendors, two architecture families). Four findings replicate across all three: (1) zero‑shot composition and sequential re‑calibrated editing diverge by 61‑71% of the edit magnitude; (2) cut order is not interchangeable, and the effect scales with concept overlap ‑‑ in one charge the order of cutting two Paris landmarks decides whether an uninvolved third concept survives; (3) the survivors' layer‑input covariances ‑‑ the method's own sufficient statistics, read as strain gauges ‑‑ drift monotonically with every further cut, in every surviving concept, in every charge; (4) erased knowledge partially returns under subsequent unrelated cuts. Appendix F's commutative‑manifold hypothesis is thereby falsified for sequential editing; the single‑edit results of the original paper are untouched. For unlearning‑as‑compliance: erasure certified today does not certify the artifact after its next edit.
Authors:Jinwei Kong, Runqi Meng, Fanyi Wang, Wentao Qiu, Haotian Hu, Yongjian Zhou, Zhenhua Ge
Abstract:
Sparse Mixture‑of‑Experts (MoE) models expand foundation model capacity through conditional expert activation, but their full expert pools remain difficult to deploy under limited accelerator memory. Although expert offloading alleviates memory pressure by moving inactive experts to host memory or storage, it introduces a routing‑dependent transfer bottleneck: required experts are known only after native top‑\(K\) routing, which serializes routing, expert loading, and expert execution during inference. To address this bottleneck, we propose SpecPrefetch, a parameter‑efficient prefetching framework for offloaded MoE inference. SpecPrefetch uses a shared lightweight adapter to predict next‑layer expert candidates only for asynchronous transfer, while the frozen native router still determines the final executed experts. By separating transfer prediction from execution routing, SpecPrefetch reduces exposed expert‑loading latency without changing pretrained routing semantics, so prediction errors affect transfer efficiency rather than model outputs. In addition, a window‑aware scheduler prioritizes feasible transfers under cache and bandwidth constraints. Across Qwen3‑VL‑30B‑A3B and DeepSeek‑VL2‑Tiny, SpecPrefetch achieves the best average expert recall in 9 out of 10 model‑benchmark settings with substantially fewer trainable parameters than learned predictor baselines. On a Snapdragon 8 Elite device, SpecPrefetch further improves decoding throughput by up to \(20%\) over a compute‑optimized offloading runtime, demonstrating practical benefits for storage‑constrained MoE deployment. The code and model weights are available at https://github.com/wei390/SpecPrefetch.
Authors:Tengfei Lyu, Jindong Han, Hao Liu
Abstract:
Nuclear radiation, the energy released during atomic decay, poses persistent risks to public health and the environment, and concerns have only grown since the Fukushima accident and the recent commencement of treated‑water discharge. Modern monitoring networks now record radiation levels and accompanying weather conditions at thousands of stations, opening the door to nationwide forecasting that can inform emergency response, agricultural advisories, and routine public‑safety decisions. However, turning this abundance of monitoring data into reliable forecasts is difficult for three reasons. First, the time series at each station are highly non‑stationary, shaped by radioactive decay, weather variability, and irregular human interventions. Second, monitoring stations are severely unevenly distributed in space. Roughly 78% of Japan's stations sit in less than 6% of the country, clustered near Fukushima, which breaks the assumptions of standard graph‑based models. Third, radiation co‑evolves with heterogeneous context such as wind, temperature, and humidity through atmospheric transport processes that purely data‑driven models struggle to capture from observations alone. In this study, we introduce NRFormer+, a spatio‑temporal Transformer for nationwide nuclear radiation forecasting. NRFormer+ couples non‑stationary temporal attention and density‑adaptive spatial attention with a new atmospheric diffusion module that estimates how meteorology drives radiation dispersion and injects this physical signal into the network as an architectural prior. NRFormer+ delivers state‑of‑the‑art accuracy on both datasets across all 13 baselines, reducing sudden‑change MAE by up to 19.1% over the strongest baseline at comparable inference latency. Our code and datasets are publicly available at https://github.com/tfeilyu/NRFormer_Plus.
Authors:Bingxian Wu, Yu Zhang, Zonghao Guo, Tang Liu, Chen Qian, Yuxiang Lu, Xingbo Du, Yanghao Li, Yidan Zhang, Chi Chen, Ling Yao, Maosong Sun
Abstract:
Geoscience research requires complex analysis and domain expertise, with remote sensing (RS) observations as a key foundation. However, existing RS agents built on general‑purpose LLMs remain largely domain‑agnostic, resulting in brittle and error‑prone workflows. Moreover, these failures are seldom consolidated into a reusable experience for subsequent analyses. To address this issue, we introduce RSMeM, a knowledge‑enhanced memory evolution mechanism that bootstraps RS agents with pre‑distilled domain knowledge and iteratively integrates online experience for robust multi‑step tool execution. RSMeM is composed of two components: (i) Hierarchical Knowledge Grounding, which performs taxonomy‑aware retrieval over a hierarchical domain corpus to guide planning and tool selection; and (ii) Failure‑Aware Experience Refinement, which distills failure‑annotated tool‑use traces into reusable constraints for next‑round tool execution. By iteratively employing these two processes, RS agents can evolve to absorb task‑level domain knowledge and effectively translate it into instance‑level execution experience. Extensive experiments on EarthBench demonstrate that RSMeM consistently improves tool‑use performance and end‑to‑end answer across a diverse set of LLM backbones. Notably, RSMeM achieves a 6% accuracy improvement on DeepSeek‑V3.2 with less than 1% additional experience tokens, demonstrating the strong knowledge density of our distilled experience. Our code is available at https://github.com/AI9Stars/RSMeM
Authors:Vaibhav Balloli, Carissa Samuel, Samia Abdelnabi, Alex Peahl, Elizabeth Bondi-Kelly
Abstract:
Prenatal care is an important preventive service designed to improve outcomes for pregnant individuals. The American College of Obstetricians and Gynecologists (ACOG) recently introduced guidelines advocating tailored prenatal care, called PATH (Plan for Tailored Healthcare). We present PATHFinder Agent(Planner for Appropriate Tailored Healthcare), an end‑to‑end conversational agentic system that gathers patient health and social context through structured dialogue, curates individualized prenatal care plans aligned with PATH guidelines, and surfaces community resources from Michigan 211. The system features a four‑stage workflow spanning patient intake, dynamic interaction, plan synthesis, and clinician oversight. We evaluate frontier large language models (LLMs) on expert‑curated rubrics across five clinical dimensions, finding that GPT‑5.2 achieves the highest average score (77.6%) while identifying key gaps in antenatal testing recommendations. We discuss future validation through human participant studies and randomized controlled trials.
Authors:Gi-Hun Lee, Joong Yull Park
Abstract:
Large language models (LLMs) can give different answers to the same decision problem across runs, and reverse a decision when their own prior answer returns as context. We ask whether this instability can be measured and partially reduced without changing model weights. We test the Cognitive Kernel Model (CKM), a prompt‑level state‑enforcement layer. Before deciding, the model must separate its input into three epistemic roles: Fact (given or verifiable), Heuristic (inferred or assumed), and Emotion (evaluative or priority signal). CKM adds no capability; it forces the model to track what kind of information it uses before acting. Formally it maintains a structured state S_t = F_t, H_t, E_t updated by a transition function. We evaluate CKM on Korean‑language decision scenarios (ambiguity, ethical conflict, resource allocation, error handling) across 26 LLMs from four vendors and 37,403 observations, via four core experiments, a 4‑arm ablation, a 5‑arm sham‑restriction ablation, and a temperature probe. Findings: (1) CKM reduces repeated‑output variability (random‑effects Hedges' g=1.09, 95% CI [0.83, 1.35], 31 model pairs); (2) state persistence cuts the decision‑flip rate by 82% in newer models (g=1.52); (3) the effect is not JSON formatting alone (value‑only recomputation, g=2.24); (4) intrinsic randomness under fixed anchor states is negligible; (5) the advantage grows under sampling stochasticity (g=2.87 at temperature 0.7); (6) a sham ablation attributes about 45% of the gain to structural scaffolding and 55% to Fact/Heuristic/Emotion content, and CKM is the only arm that both raises consistency and reduces flipping. CKM does not improve reasoning correctness. The narrower result: behavioral consistency is measurable, varies across models, and is partially improvable by forcing models to separate facts, assumptions, and evaluative signals before deciding.
Authors:Joshua Brodsky, Dhravid Kumar, Savini Kashmira, Jayanaka Danatanarayana, Jason Mars, Krisztian Flautner, Lingjia Tang
Abstract:
Machine learning models are increasingly embedded in everyday software, and most of their runtime is spent in a small set of compute kernels such as matrix multiplication, convolution, and normalization. Optimizing these kernels is one of the most direct ways to reduce latency and cost, but it has traditionally required expert engineers to hand‑write low‑level GPU code. Agentic systems built on large language models (LLMs) can now generate and optimize kernels with far less human effort, yet existing tools are largely evaluated on randomly generated tensors and isolated kernels, emit standalone CUDA code that developers must manually reintegrate, mostly target only LLM PyTorch models, and offer limited support for inspecting and debugging results. We present Kernel Forge, an open‑source, end‑to‑end agentic harness that accepts any unmodified PyTorch model in place. Kernel Forge supports vision, diffusion, and LLM workloads, uses Monte Carlo Tree Search (MCTS) to explore multiple optimization paths rather than a single linear refinement chain, and ships with a graphical user interface for monitoring progress, inspecting candidate kernels, and debugging failures. We evaluate Kernel Forge on four PyTorch models spanning vision, diffusion, and LLM workloads on an NVIDIA DGX Spark with GB10 GPU. With only 50 optimization iterations per kernel, it optimizes 14 kernels to outperform PyTorch eager mode, reaching 1.52× on adaptive\_avgpool2d in ResNet‑50, 1.70× on group\_norm in Stable Diffusion 3.5 Medium, 2.83× on softmax in Gemma 4 E2B, and 1.54× on softmax in Qwen 3.5 35B‑A3B.
Authors:Hanxi Li
Abstract:
Retrieval‑augmented generation (RAG) has made substantial progress in extending the memory of large language models (LLMs), and recent advances have further pushed RAG from pure text settings toward multimodal scenarios. In the document understanding domain, document visual question answering (DocumentVQA) has evolved from question answering over a single document to retrieval‑and‑generation pipelines over large‑scale document collections. However, a benchmark specifically designed for Chinese large‑scale document retrieval and question answering is still lacking. To bridge this gap, we introduce CHaystack, a new Chinese DocumentVQA benchmark that covers four document categories, namely academic papers, advertisements, web pages, and real‑world photographed documents, enabling a more comprehensive evaluation of DocumentVQA systems. In addition, we present CDocRAG, a Chinese DocumentVQA system that uses a VLM‑based relevance filter to verify retrieved document images before answer generation. We evaluate representative open‑source embedding and generation models on CHaystack. The results reveal a clear contrast in category‑wise strengths: Qwen‑family models perform best on text‑rich documents such as webpages and papers, whereas other models only achieve competitive results on visually rich categories such as advertisements and degrade sharply on text‑dense documents. For retrieval, Qwen3‑VL reaches 71.91 Recall@1 while the best non‑Qwen model achieves only 14.40. These results indicate that the core challenge of CHaystack lies in Chinese textual encoding, and that Chinese large‑scale DocumentVQA still leaves substantial room for improvement. Our code and dateset is available at https://github.com/hanxi19/CHaystack.
Authors:Hangjie Yuan, Yichen Qian, Zhiwei Tang, Xianzhe Xu, Lirong Wu, Sicheng Yang, Jinwang Wang, Pengju Wang, Zhitao Zeng, Yizeng Han, Yan Xing, Shengxuan Luo, Tao Feng, Qing Xie, Weigen Yao, Yi Yang, Zuozhu Liu, Jiasheng Tang, Shaocheng Wang, Jitao Wang, Jiahong Dong, Weihua Chen, Feng Xu, Fan Wang
Abstract:
Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision‑centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists' clinical practice and provide an accurate, fine‑grained and factualness‑driven assessment. In this paper, we introduce ClinFusion, a vision‑centric MLLM designed for holistic medical understanding that systematically addresses these limitations. We propose a compositional and cascaded vision encoder architecture featuring a Cascade Spatial‑Aware Locality Fusion operator that unifies diverse 2D and native 3D medical image understanding within a fused encoder. We further introduce a vision‑grounded evaluation framework, including MedIF‑Bench for instruction‑following assessment and a region‑of‑interest‑grounded method for clinically aligned and factualness‑driven report generation evaluation. We show that ClinFusion sets a new state‑of‑the‑art across a comprehensive suite of 2D and 3D multimodal medical benchmarks‑‑‑spanning visual question answering, report generation, and instruction following‑‑‑as well as textual medical tasks, outperforming leading open‑source medical MLLMs (e.g., Hulu‑Med, Lingshu) on 20 out of 24 benchmarks and demonstrating multimodal capabilities better than powerful proprietary models such as GPT‑5.2 and Gemini‑3‑Flash on 13 out of 16 benchmarks, and can be further augmented with agentic tool use for retrieval‑augmented and tool‑assisted clinical workflows. A blinded evaluation by board‑certified radiologists confirms that ClinFusion produces the highest‑ranked reports, and validates our RoI‑grounded metric as achieving the strongest correlation with expert judgment among all automatic evaluation metrics examined.
Authors:Weijie Xia, Stefanie Horian, Hanyue Huang, Queena K. Qian, Jie Yang, Pedro P. Vergara Barrios
Abstract:
Recent studies use Large language models (LLMs) to simulate human opinions and decisions by prompting models with demographic, attitudinal, or persona‑based descriptions. Yet such simulations rarely model the practical, cognitive, or social frictions that shape how people respond to policy interventions. Perceived transaction cost (PTC) provides a useful lens for modeling the practical frictions that shape policy responses, such as information burden, administrative effort, coordination demands, and perceived uncertainty. We use this lens to develop a friction‑aware persona modeling approach for LLM‑based simulation. In the context of energy‑efficient renovation (EER), tenants are represented not only by who they are demographically, but by how they perceive the costs, benefits, barriers, and uncertainties associated with proposed renovation plans. Using survey data collected from 1,068 citizens in the Netherlands, comprising approximately 40,548 survey question and answer pairs, we compare prompt‑only and fine‑tuned settings across GPT‑3.5‑turbo, Ministral‑8B‑Instruct, and Llama‑3.1‑8B‑Instruct, and evaluate supervised fine‑tuning (SFT) and Group Relative Policy Optimization (GRPO) for local open‑weight models. Results show that incorporating PTC‑based personas and reasoning consistently improves model performance across both prompt‑only and fine‑tuned settings, suggesting that PTC‑based persona design provides a useful bridge between institutional policy theory and interpretable LLM‑based policy simulation. Code is available at https://github.com/xiaweijie1996/socialagent.
Authors:Hengyuan Zhang, Jingna Sun, Meiguang Jin, Junfeng Ma
Abstract:
Production‑ready audio‑driven avatar generation requires efficient inference without sacrificing fidelity or motion expressiveness. However, existing acceleration methods often compromise quality through restrictive architectural choices, such as causal attention and short temporal horizons, or by reducing model capacity and resolution. Without such compromises, we propose AptAvatar, a 14B‑parameter long‑form audio‑driven avatar generation framework that delivers fast and expressive inference. For efficiency in production‑level applications, AptAvatar addresses the extreme two‑step generation challenge. To bridge the gap between the multi‑step teacher model and the two‑step student model, we introduce Endpoint‑Anchored Distribution Distillation. It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory‑endpoint distribution defined from a frozen pretrained 4‑step bridge generator. This provides an attainable endpoint‑level anchor for the evolving two‑step student. To improve long‑horizon consistency, we further introduce Self‑Generated History Replay, which reuses cached outputs from earlier generator checkpoints as history conditions during chunk‑wise training. This approximates inference‑time conditioning on self‑generated histories without costly online rollouts, mitigating quality degradation from accumulated history errors. Extensive experiments demonstrate that AptAvatar generates vivid 720p long‑form avatar videos with only 2 NFEs, achieving a 60x speedup while preserving visual fidelity and long‑horizon identity. Code is available at https://github.com/TaoLiveAIGC/AptAvatar
Authors:Hai Jiang, Yixian Zou, Binbin Liang, Boqian Liu, Fanman Meng, Shuaicheng Liu
Abstract:
Real‑time deployment of Vision‑Language‑Action (VLA) policies necessitates asynchronous execution, wherein subsequent action chunks are computed concurrently with the execution of the current chunk, leading to prediction‑execution misalignment and manifesting as inter‑chunk discontinuities. Existing methods either superficially smooth chunk boundaries, require costly policy optimization, or exclusively forward‑predict proprioceptive states yet neglect critical visual observations. In this paper, we propose FutureRTC, a plug‑and‑play adaptation framework that predicts execution‑time observations and states for asynchronous VLA control without modifying the underlying policy. Specifically, FutureRTC features a state correction module to compensate for the discrepancy between rolled‑forward and actual execution‑time proprioceptive states and an observation prediction module that forecasts execution‑time visual representations by leveraging robot motion as an explicit physical prior through motion‑aware feature transport and reconstruction. Furthermore, we introduce a policy consistency loss to align the action chunks generated from predicted contexts with those produced under the expected execution‑time inputs of the VLA policy. Extensive experiments across simulated and real‑world environments demonstrate that FutureRTC achieves superior robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.
Authors:Tapan Parikh
Abstract:
Appending a two‑word confirmation tag to a decision question ‑‑ "Is X the better choice?" versus "X is the better choice, right?" ‑‑ changes whether a language model endorses the choice. We measure this tag effect on 20 frozen, ground‑truth‑free decisions between two defensible options, counterbalanced so a model's own preferences cancel, scored by exact match on clamped yes/no replies ‑‑ no LLM judge, no embeddings. Across 45 models the effect spans +32% to ‑32% ‑‑ a 64‑point swing on one word ‑‑ with 5 models significantly sycophantic and 17 significantly resistant (BH‑FDR q=.10). The sign is a clock: within model families the effect crosses from positive to negative as generations advance (GPT +4 to ‑28; Claude +7 to ‑32; Qwen and Grok likewise), roughly ‑6 points per year, a reversal robust to vendor tier; one lineage (DeepSeek) never crosses, and two releases during the study window (Claude Opus 5, Gemini 3.6 Flash) land on the trend out‑of‑sample. A full‑panel ablation localizes the resistance as a double dissociation: a synonym tag reproduces each model's response almost exactly (r=0.89), while planting the same preference without a tag produces resistance in no resistant model (stance effects +6 to +49; r=0.23 with tag effects). The resistance is keyed to the surface construction of a tacked‑on agreement bid, not the user's stance ‑‑ a pattern‑match, not a principle. And the tag's polarity matters more than its presence: swap one word ‑‑ "X is the better choice, maybe?" ‑‑ and agreement rises above the neutral baseline in 45 of 45 models (+19.6 points), with ten models affirming both mutually exclusive options at 90‑100%. Agreement tracks how sure the user sounds, in opposite directions at the two poles. The instrument is one word, one dollar, and judge‑free; run per release, it reads the field's anti‑sycophancy training directly off model behavior.
Authors:Stefan G. Creadore
Abstract:
Large language model research agents can connect literature retrieval, analysis code, and manuscript preparation, but coherent output does not establish scientific validity. We developed Plato‑Bio, a biology‑routed extension of the open Plato/Denario architecture that couples explicit workflow states with provenance records, citation checks, claim‑to‑evidence links, scoped file writes, and publication gates. A source audit identified and repaired three defects that could distort evaluation: loss of task domain in the default factory, omission of declared method signals from scoring, and evidence sidecars that lacked the drafted‑claim denominator. On the current clean revision, the full Python suite completed with 931 passes, six skips, and no failures or errors; targeted biology, genomics, evidence/citation, and adversarial‑safety suites likewise completed without failure. We evaluated two narrow use cases. In a frozen historical rediscovery task, independent pre‑1986 literature bridges ranked the later‑studied relation between fish oil and Raynaud phenomenon first; TF‑IDF ranked it second and corpus frequency third. This single curated task measures retrospective ranking, not prospective discovery. In a separate comparison of AlphaFold models with experimental structures for 15 human proteins, 11 targets had high‑confidence‑core C‑alpha RMSD below 1 Angstrom (median 0.501 Angstrom). Four targets exceeded 2 Angstrom, and confidence masking reduced the SUMO1 discrepancy from 16.61 to 2.58 Angstrom over 74 residues. The workflow emitted 27 traceable discrepancy regions, all retained as unvalidated hypotheses. Plato‑Bio therefore provides reproducible software contracts and auditable screening baselines; broader claims of agent efficacy or biological novelty require preregistered evaluation, independent review, and prospective validation.
Authors:Pei Liu, Nan Zheng, Lang Zhang, Daojie Peng, Yanan Zhang, Feilong Kong, Mingyue Feng, Jiachao Liu, Yaonong Wang, Qifeng Chen, Jun Ma
Abstract:
World Action Models (WAMs) have emerged as a powerful paradigm for embodied intelligence, yet the prevailing reliance on pixel‑level video generation creates a fundamental bottleneck. Forcing models to reconstruct task‑irrelevant visual details dissipates representational capacity and renders policies vulnerable to visual distractors. In this paper, we propose LeapBot‑WA, which establishes a novel Predictive‑Latent paradigm for WAMs by operationalizing the Joint‑Embedding Predictive Architecture (JEPA) as a World‑Anchor. Departing from the traditional reliance on visual synthesis, LeapBot‑WA shifts the core of world modeling to Predictive Semantic Alignment, extracting abstract physical dynamics directly within a latent foundation space. To bridge the modality gap between non‑Gaussian predictive features and diffusion priors, we introduce the Isotropic Semantic Autoencoder (ISAE), which reshapes the anchor's latent space into a diffusion‑friendly manifold to prevent off‑manifold drift. Furthermore, we design an Asymmetric Mixture‑of‑Transformers (MoT) architecture. During training, an Anchor Diffusion Transformer acts as a privileged dynamics expert to guide the Action Diffusion Transformer; at inference, this heavy dynamics branch is pruned, enabling zero‑overhead execution. LeapBot‑WA achieves state‑of‑the‑art performance among predictive models on LIBERO and matches top‑tier generative WAMs on RoboTwin 2.0 without requiring large‑scale trajectory pre‑training. It further demonstrates superior zero‑shot robustness to unseen environments and successful real‑world transfer, establishing a highly efficient and robust latent‑centric paradigm for scalable robotic control. Code: https://github.com/LeapWM/leapbot‑wa.
Authors:Zach Manson, Barry C. Sanders
Abstract:
We study the impact that two miners equipped with quantum computers purpose‑built for quantum Bitcoin mining will have on the 51% attack threshold of the Bitcoin network, given that the miners are playing a competitive game against each other to be the first to mine a block. We extend an existing game‑theoretic framework for Bitcoin mining and compute the resultant payoff matrices. From these payoff matrices, we determine optimal quantum mining strategies for two non‑colluding and aggressive quantum miners with multiple opportunities at finding a valid block in an otherwise classical Bitcoin network. We show that these optimal quantum mining strategies have a negligible effect on the 51% attack threshold. The novelty of our work is the inclusion of the Aggressive Quantum Mining Strategy and the realistic approach of allowing the quantum miners to restart their search if their measurements do not yield a valid block when determining the optimal quantum mining strategies. Our result is important for evaluating quantum‑mining threats on cryptocurrencies based on Proof‑of‑Work, e.g. Bitcoin
Authors:Hao Yang, Jin Wang, Xuejie Zhang
Abstract:
Human visual reasoning typically follows a coarse‑to‑fine attention process, starting from global scene understanding and gradually focusing on question‑relevant regions. However, multimodal large language models may deviate from this pattern due to attention drift and the underutilization of visual evidence, which can lead to hallucinations. To mitigate these issues, this study proposes a Dual‑Indicator Guided Contrastive Alignment (DICA), which tracks two information‑theoretic indicators during inference: Visual Attention Entropy (VAE), which reflects the concentration of visual attention, and Output Image Correlation (OIC), which measures the dependence of generated outputs on the visual input. An abnormal increase in VAE or a decrease in OIC corresponds to different failure modes, which trigger targeted contrastive alignment to restore visual grounding. Experimental results across multiple benchmarks demonstrate that DICA consistently outperforms existing approaches and substantially reduces hallucinations, highlighting the effectiveness of indicator‑driven intervention in improving multimodal inference reliability. The code is publicly available at https://github.com/BGWH123/DICA/.
Authors:Anjie Le, Can Peng, Hongcheng Guo, J. Alison Noble
Abstract:
Machine unlearning, which aims to remove the influence of specific training data from a trained model, is a key requirement for privacy, accountability, and adaptive deployment. We argue that many unlearning methods are vulnerable to a simple clustering attack, which can recover class structure in an unsupervised manner, limiting their suitability for continual deployment where removal requests must be handled reliably on demand. To address this, we propose DECAF (DE‑Clustering for Adaptive Forgetting), a post‑hoc method that operates only on the forget set and is designed to break the cluster. DECAF combines input noise, confidence suppression, and entropy‑based output diversification to disrupt the residual feature‑space structure associated with forgotten data. On CIFAR‑10 with ResNet‑18, DECAF attains 0.10% forget‑class accuracy, 79.4% retain accuracy, and an AUS of 0.88, surpassing all other baselines. In cluster‑based analysis, it attains performance comparable to that of unlearning methods that use the full training set, while being significantly more efficient. Code: https://github.com/ale256/representation_unlearning.
Authors:Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang, Chih-Fan Hsu, Jeng-Lin Li
Abstract:
Vision foundation models have enabled strong training‑free anomaly detection (AD). However, most existing approaches rely primarily on independent local patch features, leaving the global contextual information encoded by Vision Transformers (ViTs) underexploited. In this work, we identify the dual characteristics of the ViT [CLS] token: its embedding provides anomaly‑invariant global semantic representation, while its attention maps implicitly highlight spatially abnormal regions. Building on this observation, we propose a fully automated AD framework leveraging global context to remove manual tunings. Our framework introduces (1) an automatic augmentation selection strategy driven by [CLS]‑level semantic consistency, and (2) an attention‑guided feature reweighting mechanism that dynamically adjusts patch contributions according to [CLS] attention saliency. By integrating these components over multi‑level features, our method achieves stable anomaly scoring and precise localization without training or parameter tuning. Under the one‑shot setting, it achieves Image‑AUC scores of 97.7%, 93.2%, and 84.5% on MVTec‑AD, VisA, and Real‑IAD. Using a single fixed configuration across categories, backbones, and datasets, the method establishes a new state‑of‑the‑art for plug‑and‑play, training‑free anomaly detection while maintaining strong robustness and practical scalability.
Authors:Jun Ling, Tao Huang, Junzhuo Liu, Bowen Tang, Peng Wang
Abstract:
Modern vision‑language models (VLMs) increasingly rely on dynamic or high‑resolution visual encoding, producing thousands of visual tokens that substantially increase downstream language‑model inference cost. Existing token‑reduction methods assess token utility through token‑wise importance, query relevance, coverage, pairwise diversity, or subset‑level objectives. Our key insight is to view visual token reduction through selected‑span complementarity: instead of scoring a token in isolation or through pairwise relations, we assess how much of its feature is orthogonal to the span of the already retained subset. Based on this perspective, we propose Greedy Orthogonal Token Selection (GOTS), a training‑free and query‑agnostic method. At each step, GOTS selects the token with the largest residual energy orthogonal to the current retained span. This rule exactly maximizes the one‑step augmented Gram determinant among candidate additions, giving each greedy step a precise local geometric guarantee for subset expansion. Across five high‑resolution VLM backbones from the Qwen‑VL and InternVL families and eleven diverse benchmarks, GOTS achieves higher average performance retention than the strongest evaluated baselines, and a controlled OCRBench study shows that it reduces model‑side time‑to‑first‑token after accounting for selection overhead. Code is available at https://github.com/newLLing/GOTS.
Authors:Goodarz Mehr, Sepideh Gohari, Montasir Abbas, Azim Eskandarian
Abstract:
Cooperative perception through vehicle‑to‑everything (V2X) communication can overcome the inherent physical limitations of individual autonomous vehicles, such as occlusions and limited sensor range. However, the development of robust V2X algorithms, particularly those relying on unified spatial representations like bird's‑eye view (BEV) representation, is hampered by the lack of large‑scale, multi‑modal, multi‑task datasets. Moreover, collecting and annotating a large set of synchronized, real‑world multi‑agent data is prohibitively expensive. This has resulted in a landscape where existing V2X datasets are notably limited in both size and scope. To overcome this, we introduce SimBEV2X, an advanced synthetic data generation tool built on the CARLA simulator. SimBEV2X automatically creates randomized driving scenarios to collect multi‑modal sensor data alongside various types of ground truth including 3D bounding boxes with unique track IDs, HD map information, BEV segmentation maps, and semantic occupancy voxel grids from both vehicles and RSUs. We also present the SimBEV2X dataset, the largest V2X perception dataset to date. The dataset comprises 258 scenes, each involving up to 8 connected vehicles and up to 4 RSUs across a variety of road networks. The SimBEV2X dataset is an order of magnitude larger than existing V2X datasets and contains 102,200 frames, 588,520 lidar point clouds, more than 3 million images, over 27 million bounding boxes, and a comprehensive set of other annotations. Finally, we establish a strong baseline on the SimBEV2X dataset using CoopDet3D and propose CoBEVFusion, a novel architecture that combines CoopDet3D with fused axial attention (FAX) for context‑aware multi‑agent feature aggregation, resulting in superior performance. SimBEV2X, the SimBEV2X dataset, and CoBEVFusion are available at https://simbev2x.org and https://github.com/GoodarzMehr/SimBEV2X.
Authors:Hector R. Rodriguez, Jiechen Huang, Wenjian Yu
Abstract:
We present Flash‑CNNCap, a CNN‑based capacitance extractor that reformulates full‑matrix capacitance prediction as image‑to‑image regression over spatial contribution maps. Prior scalar CNN‑based extractors require O(n^2) forward passes to recover all pairwise capacitances in a window with n conductors. Flash‑CNNCap replaces the scalar target with dense contribution maps: a total‑capacitance model and a master‑conditioned coupling model each predict a spatial map that is reduced to conductor‑level values through mask aggregation, cutting full‑matrix reconstruction to O(n) passes. The resulting totals and symmetrized pairwise couplings define the corresponding Maxwell‑style capacitance matrix under the standard off‑diagonal sign convention. The maps are learned from conductor‑level labels without per‑pixel supervision. An ablation study over 13 model configurations selects a U‑Net that matches ResNet baselines on total capacitance (1.5‑3.1% MARE) and achieves the strongest coupling accuracy (3.0‑4.6% MARE) across all evaluated CapBench subsets, with a 17.5× full‑matrix speedup on windows containing 134 conductors on average. A deployed pipeline reads Design Exchange Format (DEF) geometry and writes Standard Parasitic Exchange Format (SPEF) output, processing 1,024 windows in 51.23 seconds with a 4.4× speedup over OpenRCX on the same benchmark. Code and trained models are available at https://github.com/THU‑numbda/flash‑cnncap.
Authors:Kartik Teotia, Helge Rhodin, Hyeongwoo Kim, Marc Habermann, Christian Theobalt
Abstract:
Facial movement and expression are central to face‑to‑face communication, conveying turn‑taking, attention, agreement, and engagement alongside speech. While speech‑driven facial animation has made strong progress in lip synchronization and audio‑conditioned motion generation, most methods treat conversational behavior as an emergent byproduct of audio, or expose only coarse sequence‑level affect control. As a result, key non‑verbal channels such as gaze contact and aversion, rhythmic head motion, and emotion remain difficult to explicitly control. We present STEER, a controllable 3D dyadic motion prior for reactive conversational head avatars. STEER factorizes conversational behavior into explicit controls for gaze, head rhythm, and emotion, allowing users to steer how an avatar listens, reacts, and engages with a conversation partner. Since temporally aligned annotations for these behaviors are not available in public dyadic corpora, we introduce a tracking and annotation pipeline that recovers behavioral pseudo‑labels from in‑the‑wild dyadic video. A causal flow‑matching transformer then learns partner‑aware target motion conditioned on audio, partner motion, emotion and the proposed behavioral controls. We further embed STEER in a photorealistic avatar pipeline by extending a Universal Gaussian Head‑Avatar Prior with a learned mapping from tracked parametric motion into its avatar‑driving space. This enables controllable animation of high‑fidelity Gaussian head avatars without re‑training the underlying avatar model. STEER outperforms recent dyadic motion baselines on motion quality, dynamics, and diversity, remains competitive on partner coupling, and enables gaze, head‑rhythm, and emotion edits together with an interactive live deployment. We make our code and dataset annotations available at our webpage.
Authors:Xiaochuan Li, Ryan Ming, Meng Chu, Shuai Shao, Rong Jin, Chenyan Xiong
Abstract:
Agentic tasks are inherently long‑horizon and multi‑turn, constantly accumulating context through interactions with the environment. Existing context compression methods inevitably incur information loss and are triggered by rigid heuristic rules, leaving them misaligned with the agent's evolving reasoning focus. We propose Agentic Context Management (ACM), a framework that equips agents with purpose‑built context editing tools for lossless context management. Inspired by the interaction between short‑term and long‑term human memory, the agent autonomously decides when to compress its context, offloads discarded content to an external memory system, and queries it on demand for later retrieval. Building on this framework, we further develop a post‑training pipeline that constructs high‑quality demonstrations of context management and improves model performance on both agentic search and coding tasks. Further analysis reveals that effective context management reduces peak token pressure, enables extended explorations, and yields more consistent solutions across independent trials. Code, data, and model checkpoints are available at https://github.com/lixiaochuan2020/agentic‑context‑management.
Authors:Sietse Schelpe
Abstract:
Improving a language model today means retraining it: enormous compute, a new opaque model each cycle, non‑deterministic output. We take the opposite path: the model stays frozen, and a persistent memory of verified solutions grows beside it. Once a problem family is solved and has passed an independent verification step that never consults the answer key, every new instance of that family is answered at zero generation tokens, bit‑exact, deterministically. Across 180 fresh instances spanning nine problem families, four architectures from four vendors ‑ dense and mixture‑of‑experts ‑ each score 180/180 at zero generation tokens per answer: execution‑bound capability decoupled from parameter scaling. A negative control attributes the capability fully to the memory: emptied, it solves nothing. The same verify‑before‑store contract holds for open‑ended reasoning: 88/88 consistency‑gated acceptances across all four models, machine‑checked formal proof, and reasoning‑method transfer at 77/80. Memory selection takes 1.4 microseconds; a full reuse completes in 6‑23 ms at 36 mWh. Approximate similarity retrieval selects the wrong item 94.3% of the time on a 4,500‑item verified store where exact addressing makes zero errors. The store also serves as working context at a scale no shipped engine matches: a 6,000,000‑token movable window on a single 46 GB GPU at flat memory, where vLLM stops at 30,399 tokens and SGLang silently truncates past 32,000. On published benchmarks, frontier models remain far ahead of any 12B at raw from‑scratch reasoning; on everything this system has solved and verified, the comparison inverts: a frontier API call pays a fresh generation pass on every query, forever, while verified reuse costs zero tokens and returns the identical bits every time. A public testbench with free, rate‑limited access accompanies this report: https://corbenic‑galahad‑bench.hf.space
Authors:Qinsi Wang, Jing Shi, Huazheng Wang, Kun Wan, Yiran Wu, Bo Liu, Qingyun Wu, Hai Helen Li, Yiran Chen, Handong Zhao, Wentian Zhao
Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has driven recent progress in reasoning‑oriented large language models (LLMs) by enabling large‑scale optimization. However, its applicability remains largely limited to domains such as mathematics and coding, where correctness can be deterministically verified. Open‑ended tasks instead often rely on human preferences, reward models, or LLM‑based judges, introducing evaluation bias, judge capability bottlenecks, and additional inference costs.Drawing on the principle of self‑supervised learning, which constructs pretext tasks to derive supervision from the data itself, we propose Reinforcement Learning with Self‑Verifiable Rewards (RLSVR), a task‑transformation‑based training paradigm for extending RLVR to open‑ended tasks. RLSVR transforms open‑ended tasks into verifiable proxy environments whose internal rules and interaction outcomes automatically generate reward signals. We instantiate RLSVR with SpyRL, a multi‑agent self‑play environment inspired by Who Is the Spy?. Agents receive asymmetric information, complete the same target task, and vote to identify a designated spy. Because the spy identity is predetermined, voting outcomes provide fully verifiable rewards, while successful identification remains closely related to output quality. Experiments on text summarization, creative writing, and mathematical reasoning show that SpyRL outperforms existing self‑improvement methods on non‑verifiable tasks and yields consistent gains on verifiable reasoning tasks. These results demonstrate that task transformation can extend scalable RLVR‑based self‑improvement beyond inherently verifiable domains. Models and code have been released at https://github.com/wangqinsi1/SpyRL.
Authors:Chi Phan, Tianyi Zhang, Yufeng Wu, Qiaochu Xue, Jiajie Zhang, Linghan Cai, Zeyu Liu, Sudong Wang, Yueming Jin, Dan Hu
Abstract:
Pathological diagnosis is inherently multi‑scale, requiring the integration of global tissue architecture at low magnification with cellular morphology at higher magnification. However, existing pathology benchmarks and vision‑language models (VLMs) are still largely developed under single‑scale settings, limiting their ability to learn clinically meaningful multi‑magnification reasoning. Moreover, naively constructed visual question answering (VQA) tasks may be susceptible to text‑only or superficial visual shortcuts, leading to unreliable assessments of visual understanding. To address these limitations, we introduce a benchmark and training framework for shortcut‑resistant cross‑scale pathology reasoning. We design an Adversarial Text‑only Screening strategy for semantic reasoning questions and a Structure‑controlled Distractor Sampling strategy for visual grounding questions, encouraging models to rely on cross‑scale visual evidence. Based on this pipeline, we construct PathScale‑VQA, a high‑quality cross‑scale pathology VQA benchmark with 10,373 multiple‑choice questions grounded in 1,368 diagnostic paths across multiple magnification levels. Building on the semantic reasoning set, PathScale‑R1 is optimized through Difficulty‑driven Reasoning Distillation supervised fine‑tuning followed by reinforcement learning with a Scale‑aware Reasoning Structure reward, which encourages the use of evidence across magnifications. Extensive experiments demonstrate state‑of‑the‑art performance of PathScale‑R1 on cross‑scale reasoning tasks and effective transfer to conventional single‑scale pathology VQA. Our code is available at https://github.com/iMVR‑PL/PathScale‑R1.
Authors:Chenghao Wu, Kesha Ou, Xiaolei Wang, Bowen Zheng, Bingqian Li, Enze Liu, Wayne Xin Zhao, Weitao Li, Long Zhang, Sheng Chen, Ji-Rong Wen
Abstract:
Recommender systems have become integral to navigating the modern digital ecosystem. Yet most deployed systems remain confined within single‑platform boundaries, observing localized interaction traces and ranking items from isolated candidate spaces. This design is poorly suited to real‑world tasks that unfold through searches, content consumption, and comparisons across multiple information sources. Claw‑style personal agents, with persistent access to authorized cross‑platform context, create an opportunity for recommendation to operate around the user rather than any single platform. In this paper, we introduce Claw‑native recommender systems, a new paradigm that moves beyond platform‑local ranking to produce unified, complementary recommendation slates spanning diverse sources and content forms. To instantiate this paradigm, we present ClawRec, the first recommender system designed to operate natively in this environment. ClawRec maintains an evidence‑linked, temporally structured user state that connects cross‑platform behaviors with cross‑source recommendations. It organizes retrieval around functional source roles and selects candidates according to their marginal utility, producing non‑redundant slates aligned with the user's active task. To enable rigorous evaluation, we introduce ClawRec‑SimBench, a benchmark constructed from sequences of concrete life events and cross‑platform behavior trajectories. Experiments show that ClawRec outperforms the strongest baselines, achieving an NDCG@20 of 0.6134 (+0.1126) and a Hit@20 of 0.6944 (+0.0854), while also improving user state quality and temporal alignment. Our code and dataset are available at https://github.com/RUCAIBox/ClawRec.
Authors:Haobo Wang, Baoli Sun, Anqi Zou, Dongsheng Huang, Zelin Lv, Ning Wang, Rui Li, Dongzhan Zhou, Weiyu Guo, Zhihui Wang, Wanli Ouyang
Abstract:
The deployment of embodied agents in self‑driving laboratories could accelerate scientific discovery, yet their reliability is constrained by the irreversible and safety‑critical nature of chemical experiments. Progress is further hindered by scarce failure data and the lack of fine‑grained evaluation protocols. To address these challenges, we introduce LabRobFail, a failure‑centric framework for learning and evaluating robotic failure analysis in chemical laboratories. LabRobFail‑Sim injects controllable failures at the control, physics, and semantic levels, enabling the construction of LabRobFail‑Data, which contains over 20,000 trajectories across 70+ task scenarios, five failure categories, and 11 fine‑grained failure types. LabRobFail‑Bench evaluates six capabilities spanning task understanding, failure detection, temporal localization, severity assessment, failure classification, and actionable correction. We further develop LabRobFail‑VLM, a domain‑specialized vision‑language model that generates structured failure diagnoses and recovery instructions. On seen environments, it achieves 92.58% failure‑detection accuracy and 85.58% temporal‑localization accuracy, substantially outperforming general‑purpose VLMs. When integrated as a real‑time supervisor, it improves downstream VLA task success rates by 10‑20 percentage points, demonstrating the value of fine‑grained failure understanding for closed‑loop recovery and reliable laboratory autonomy. Our code and data are available at https://github.com/Su‑ISE‑2001/SciRobo
Authors:Wendi Deng, Hang Du, Guoshun Nan, Haokun Tian, Jiaqi Yu, Xinlei Cao, Jaile Li, Jingfeng Chen, Ling Deng, Ting Li, Hao Yang, Jun Liu, Xudong Jiang, Sicong Leng
Abstract:
Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers. This behavior undermines interpretability and reliability, suggesting reliance on spurious shortcuts rather than faithful reasoning. Although efforts have explored step‑level supervision, distinguishing decisive steps from redundant ones remains challenging. We propose O^2‑CritiCuRL, a novel curriculum reinforcement learning framework that introduces critical‑step awareness through an iterative offline‑online paradigm. In the offline stage, O^2‑CritiCuRL conducts multi‑rollout analysis over step‑annotated trajectories to estimate step‑level importance, allowing the framework to distill critical reasoning steps and filter out redundant ones. In the online stage, we employ a progressive step‑level reinforcement learning strategy, where truncated chains guide the model to infer missing steps and refine its reasoning, thereby sharpening its focus on critical steps and overcoming the limitations of static supervision. Extensive experiments on multimodal reasoning benchmarks show that our method achieves state‑of‑the‑art performance while delivering superior training and inference efficiency. Code is available at https://github.com/kk0013/CritiCuRL.
Authors:Stylianos Ploumpis, Jan Bednarik, Gaspard Zoss, Ruslan Guseinov, Luca Prasso, Prashanth Chandran, Oliver Boyne, Vasileios Choutas, Timo Bolkart, Daoye Wang, Menglei Chai, Di Qiu, Sebastian Winberg, Gilles Rainer, Lewis Bridgeman, Delio Vicini, Jérémy Riviere, Yannick Boetzel, Alexander Koumis, Jay Busch, Cynthia Herrera, Jacob Still, Scott Ysebert, Peter Lincoln, Sergio Orts Escolano, Christoph Rhemann, Erroll Wood, Thabo Beeler, Stefanos Zafeiriou
Abstract:
Parametric models of the human head are essential tools traditionally used in computer vision and graphics for animation, rendering, and reconstruction. More recently, they serve as crucial conditioning signals within generative large vision models, allowing for tight spatial control of generated imagery. However, existing publicly available models are typically limited in anatomical scope, modeling only outer geometry while ignoring intra‑oral and ocular structures, and frequently suffer from reduced geometric quality stemming from low‑fidelity input datasets. In this report we introduce a new parametric model dubbed Generative aNthropometric Model (GNM), named as a homophone of the human genome. GNM encompasses the head, face, neck, eyeballs, teeth, and tongue, and it is built on an extensive database of high‑resolution 3D scans combined with high‑quality anatomy specific artist‑made samples. This report details the data provenance, the model architecture including the specialized sub‑models for the ocular and intra‑oral structures, and shows its SotA performance on fitting target 3D face scans. To foster community innovation, the complete GNM framework is made publicly available.
Authors:Jianing Li, Yunjian Zhang, Haiqian Han, Kangyao Huang, Xiangyang Ji
Abstract:
Conventional frame‑based imaging for active stereo systems has encountered major challenges in fast‑motion scenarios. However, how to design a novel paradigm for ultrafast depth sensing remains an open issue. In this paper, we propose a novel problem setting, namely active event‑based stereo vision, which attempts to integrate binocular event cameras and an infrared 2D pattern projector for high‑speed dense depth sensing. Technically, we first build a stereo camera prototype system and present a real‑world dataset with over 21.5k spatiotemporal synchronized labels at 15 Hz, while also establishing a realistic synthetic dataset with stereo event streams and 23.8k synchronized labels at 20 Hz. Then, we propose ActiveEventNet+, a lightweight yet effective event‑based stereo matching neural network that learns to generate high‑quality dense disparity maps from stereo event streams with low latency. Our ActiveEventNet+ mainly involves three innovations: incorporating lightweight blocks into event‑based stereo matching frameworks, designing a novel cost volume with dynamic interactions between stereo pairs, and presenting an effective temporal consistency architecture to fully use rich temporal cues in event streams. The results show that our ActiveEventNet+ outperforms state‑of‑the‑art methods while significantly reducing computational complexity. Our solution offers superior depth sensing performance compared to conventional frame‑based stereo cameras in high‑speed scenes. In particular, the lightweight ActiveEventNet enables the prototype system to achieve real‑time processing at speeds up to 150 FPS. We believe that this novel active event‑based stereo vision paradigm can provide new insights into the design of future high‑speed depth sensing camera systems. Our dataset and code can be available at https://github.com/jianing‑li/active_event_based_stereo.
Authors:Mingzhou Fan, Siyuan Xu, Mingxuan Yuan
Abstract:
Large language models (LLMs) enable autonomous agents for reasoning, planning, and tool use. Recent systems increasingly organize these agents as graphs of specialized, interconnected nodes. Although graph‑based orchestration supports flexible decomposition and coordination, it creates a key challenge: attention allocation. As workflows grow, existing approaches often execute graph components uniformly, wasting resources on irrelevant or low‑impact tasks. We introduce Attention Orchestration, a paradigm that extends Transformer‑style attention from token representations to workflow‑level agent coordination. Our framework, Adaptive Goal‑aware Attention Orchestration (AGAO), dynamically estimates agent importance based on user objectives, graph dependencies, and computational constraints. AGAO combines three components: (1) goal‑aware attention, measuring semantic relevance between user goals and agent capabilities; (2) topology‑aware attention, modeling structural dependencies in agent graphs; and (3) resource‑aware attention, allocating budgets and execution priorities across heterogeneous agents. Together, these mechanisms transform static agent graphs into adaptive systems that focus computation on goal‑critical reasoning paths. Experiments across diverse multi‑agent workloads show that AGAO improves task effectiveness while reducing unnecessary computation, latency, and token consumption compared with existing graph‑based execution strategies. Our work establishes Attention Engineering as a direction for scalable, intelligent multi‑agent systems. Code: https://github.com/MingzhouFan97/AGAO.
Authors:Keren Zhu
Abstract:
Logic synthesis has evolved from compact two‑level minimization to large multilevel flows with many interacting optimization operators. Recent work has invested substantial effort in sequencing these operators: actions are commonly treated as opaque choices in a rapidly expanding search space, while learned circuit representations and heuristic or local‑greedy orchestration provide increasingly informed ways to explore it. A central obstacle is the operator vocabulary itself. Production operators are numerous, span different representations and mathematical foundations, and expose behaviors determined by implementation‑level guards, bounds, and update order. We address this gap through agentic source analysis, using LLM agents to formulate operator‑level relations from pinned ABC and mockturtle implementations and adversarial audits to test their stated scope. The resulting certified relations yield theory‑derived operator compression: 40 deployed recipe actions collapse to a 31‑action exact Pareto cover, and source‑level conditions compile into deterministic admission gates. We integrate these gates directly into ABC Orchestrate to form TACO. Two exact gates reduce Orchestrate runtime by 11% with bit‑identical outputs on 66 circuits. In a held‑fixed integrated comparison, TACO uses fewer nodes on 14 of 16 circuits, with geometric‑mean reductions of 1.0% in nodes and 3.2% in levels, while running 2.6x faster. TACO‑max achieves an NDP geometric‑mean ratio of 0.903 on HeLO's three exact‑input rows.
Authors:Junyue Li, Ye Zheng, Yifan Chen, Zhe Sun, Xuelong Li
Abstract:
Robust object 6D pose tracking is critical for robotic systems operating in dynamic and occluded scenes. Per‑frame estimators are accurate but computationally expensive, while current trackers struggle with fast motion and complete occlusion due to their reliance on continuous visibility. To address these challenges, we present RRTrack, an efficient, recoverable object 6D pose tracker that enables robust tracking through fast motion and target disappearance‑‑reappearance. RRTrack introduces a 2D‑‑6D closed‑loop tracking strategy that integrates memory‑based video object segmentation (VOS) with 6D pose refinement. The 2D branch maintains target localization, and the 6D branch verifies geometric consistency before memory updates. In addition, a DINOv2‑based dual‑bank template matching module is developed to recover lost targets by jointly exploiting offline synthetic templates and online observation anchors while maintaining real‑time efficiency. We also introduce a synthetic RGB‑D benchmark comprising three robotic scenarios with fast motion and full occlusion. Experimental results on the synthetic benchmark demonstrate that RRTrack improves equal‑subset mean ADD‑S AR by 66.3% and ADD‑S AUC by 65.7% over FoundationPose while achieving 55.2 FPS. Real‑world experiments further validate the robustness of RRTrack under noisy sensing conditions. Project page: https://github.com/7kevin24/RRTrack
Authors:Mingxiu Cai, Zhe Zhang, Gaochang Wu, Tianyou Chai
Abstract:
The remarkable success of reconstruction‑based methods in Unsupervised Anomaly Detection (UAD) lies in their ability to identify and localize anomalies by modeling discrepancies between input images and their reconstructed counterparts. However, these approaches often struggle to capture subtle anomalies and tend to produce blurred anomaly boundaries, which significantly limits their effectiveness, particularly in complex multi‑class scenarios. To address these issues, we present XMatchAD, a novel UAD framework that reinterprets the task from a pseudo cross‑modal matching perspective. Specifically, the input and reconstructed images are treated as two complementary modalities and their matching relationships are precisely exploited for anomaly detection. First, a pre‑trained feature extractor is employed to encode discriminative representations. Second, an attention‑guided cross‑modal matching mechanism is introduced to match local inter‑modal anomaly‑related patterns while mutually refining the features. This enhances the sensitivity to anomalies with diverse shapes and subtle deviations and significantly improves the precision of anomaly detection and localization. Third, we design an adaptive frequency‑aware fusion module that further delineates sharp anomaly boundaries through the coupling of high‑frequency components from cross‑modal multi‑scale representations. Comprehensive evaluations on MVTec‑AD, VisA, and MPDD benchmarks demonstrate that our method consistently achieves superior performance, outperforming state‑of‑the‑art methods in multi‑class anomaly detection and localization. The code will be released at https://github.com/Mingxiu‑Cai/XMatchAD.
Authors:Nour Jamoussi, Ikram Dridi, Giuseppe Serra, Marios Kountouris
Abstract:
Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty‑aware prediction. Combining these two objectives remains challenging, as privacy noise can interact with the stochasticity introduced by Bayesian posterior sampling. In this work, we investigate differentially private variational Bayesian learning through the Improved Variational Online Newton (IVON) optimizer. We introduce DP‑IVON‑Gradsq, a private variant of IVON. The proposed method constructs its curvature estimate from the privatized gradient using a noise‑corrected squared‑gradient estimator, reducing the direct interaction between posterior‑sampling noise and privacy noise while preserving the Adam‑like computational efficiency of IVON. We evaluate DP‑IVON‑Gradsq on CIFAR‑10 against the standard private optimizers DP‑SGD and DP‑Adam over a range of privacy budgets. The results show that DP‑IVON‑Gradsq is competitive under weak‑to‑moderate privacy constraints, i.e., large‑to‑moderate values of \varepsilon, while degrading under strong privacy. Code is available at https://github.com/NourJamoussi/DP‑IVON‑Gradsq.git.
Authors:Oshadha Samarakoon, Dushan Herath, Ishara Ranmandala, Dilshara Herath, Roshan Godaliyadda, Parakrama Ekanayake, Vijitha Herath
Abstract:
Sky‑image irradiance studies often compare forecasting systems in which the image encoder, temporal model, fusion block, target definition, and training recipe all change together. We use a narrower protocol: the multimodal forecasting pipeline is fixed, and only the visual backbone is varied. The shared setup keeps preprocessing, clear‑sky‑index normalization, weather‑history encoding, fusion, regression head, loss, optimizer schedule, seed, and chronological split policy unchanged. We compare ConvNeXt, Swin Transformer, VMamba, Spatial Mamba, and MambaVision backbones for 10min‑ahead forecasting on Folsom and a strict matched NREL split. Forecast skill is measured against clear‑sky‑index smart persistence, and temporal‑only rows are reported as weather‑history diagnostics rather than as the main ranking criterion. On the Folsom strict split, all evaluated visual‑backbone runs improve over smart persistence. In the evaluated single‑seed strict runs, VMamba Small and Swin Base reach matched Folsom RMSE values of 65.39 W/m^2 and 65.50 W/m^2; the temporal‑only diagnostic reaches 69.51 W/m^2. On the 313‑sample NREL strict split, smart persistence remains strongest at 17.48 W/m^2, while the lowest visual RMSE is obtained by Swin Tiny at 23.76 W/m^2. These results provide a reproducible encoder comparison under one fixed multimodal operating point rather than establishing architecture‑level dominance, statistically resolved ranking, or fully optimized forecasting performance. Code available here: https://github.com/Oshadha345/irradiance_benchmark
Authors:Donghao Fu, Jingxin Li, Xue Jiang, Yihong Dong
Abstract:
Third‑party API routers have become a common layer that unifies access across increasingly diverse LLM providers. In coding‑agent workflows, high‑autonomy operation is widely adopted because it reduces interaction overhead. As a result, a third‑party API router, which sits between the agent and the upstream provider, inevitably occupies the trusted path. It can inspect and modify every request and response, yet no mechanism verifies alignment between the provider's output and the repository‑level actions ultimately executed by the agent. Consequently, client‑side permission mechanisms may become ineffective in practice. Whether this control gap produces real, hard‑to‑detect effects on software development tasks remains empirically unmeasured. In this paper, we conduct an empirical study of router‑side injection in coding agents, examining four intervention levels of increasing subtlety: Response Substitution (L1), Response Append (L2), LLM‑Polished Injection (L3), and LLM‑Polished with Distribution Alignment Injection (L4). Moreover, we develop SIDEL, a framework for trace recording, replay, injection, and defense evaluation, with a curated dataset of 400 samples. We evaluate four representative coding agents, and further evaluate whitelist‑based execution control and LLM review. Router‑side intervention substantially alters repository‑level actions and remains difficult for existing client‑side safeguards to detect. Without additional mitigations, all evaluated agents achieved a defense success rate of 0 percent across all injection levels. Client‑side mitigations and reactive reviews improve resistance but do not fully restore end‑to‑end control, motivating provider‑side output‑integrity guarantees. Our code is available at https://github.com/Riyasushin/SIDE.
Authors:Xingyang Yu
Abstract:
We present DualityCert, a symbolic verifier for candidate Seiberg‑duality claims in four‑dimensional N=1 quiver gauge theories. The verifier evaluates 't Hooft anomaly matching, superpotential R‑charge consistency, central‑charge matching, and a bounded chiral‑ring proxy. A claim that passes receives a consistency certificate, which states that no tested inconsistency was found, not that the duality is proven. We use the verifier as a repair environment for language‑model agents, which receive a deliberately broken claim and must edit it until it certifies. On a preregistered benchmark of 145 broken claims, with the analysis fixed before the first confirmatory model call, verifier‑gated retry improves final repair success over a single attempt by +8.3 percentage points (pp) on deepseek‑chat and +7.1 pp on qwen‑plus (Holm‑adjusted p<0.002). Under an equal budget of eleven attempts, the stop‑first strategy portfolio underperforms independent verifier‑filtered resampling by 10.3 percentage points on deepseek‑chat but outperforms it by 14.7 points on qwen‑plus, reversing the ordering of the two tested verifier‑exploitation policies across the two confirmatory models. On qwen‑plus, category‑level verifier feedback is worth +8.7 pp over content‑free retry, and interpretable obligation identities alone are worth +6.4 pp over structurally identical masked feedback. Neither effect is detected on deepseek‑chat. Separately, a preregistered MiniMax‑M2.5 extension again finds an iteration gain and independent verifier‑filtered resampling outperforming the strategy portfolio. Which policy is better thus differs between the two models, while every winning policy uses the same cheap certificate. The verifier, benchmark, protocol, and all per‑attempt records are released.
Authors:Xin Zhao, Yumin Liu, Zhuo Li, Weichu Zheng, Feng Zhu, Xiaokang Yang, Yaohui Jin, Yanyan Xu
Abstract:
Molecular structure elucidation from tandem mass spectra (MS/MS) is a central inverse problem in analytical chemistry. Most existing approaches to MS/MS identification remain tied to reference libraries or predefined candidate sets, whereas de novo methods aim to generate structures directly from spectra. A common de novo route predicts a molecular fingerprint from the spectrum and then decodes structures from it, enabling decoder pretraining on large molecule‑only corpora. However, this paradigm creates a training‑inference mismatch: the decoder is trained on oracle fingerprints computed from molecules, but at inference it is queried with a noisy spectrum‑induced fingerprint posterior that is typically collapsed to a single thresholded fingerprint. We introduce MS‑GPT, which recasts fingerprint‑mediated de novo elucidation as spectrum‑induced posterior querying of a conditional molecule‑language model. MS‑GPT conditions a molecule‑language model on fingerprints and formulas, then converts the spectrum‑induced posterior into a band of fingerprint queries near the oracle‑fingerprint manifold through active‑bit density calibration. Candidates sampled across this band are pooled and ranked by generation‑frequency consensus. A lightweight LoRA adapter further mitigates domain‑specific posterior bias while preserving the pretrained molecular prior. On NPLIB1 and MassSpecGym, MS‑GPT sets a new state of the art, reaching Top‑1/Top‑10 exact‑match accuracy of 29.8%/41.1% and 23.9%/28.7%, respectively. Candidate‑pool scaling shows that efficient autoregressive molecular generation continues to improve recall with a little additional inference cost. The source code and model checkpoints are available at https://github.com/VIKI623/MS‑GPT.
Authors:Wenxuan Zhang, Yuhui Wang, Donggang Jia, Xiaoqian Shen, Jian Ding, Ivan Viola, Jürgen Schmidhuber, Mohamed Elhoseiny
Abstract:
Large Vision‑Language Models (VLMs) now act as agents in interactive environments, where success requires coherent reasoning and decision‑making across turns. Although end‑to‑end training in agentic environments can improve such multi‑turn decision‑making abilities, current methods mainly rely on either token‑wise optimization over concatenated token trajectories or turn‑wise optimization with uniform within‑turn credit. In this work, we establish theoretical formulations for the two levels of optimization and derive a hybrid advantage that serves both objectives. Furthermore, with an appropriate choice of discount factor and learning target, we prove that a unified critic model can estimate values for both turn‑wise and token‑wise. As such, we propose HyGAE, an actor‑critic framework that jointly optimizes token‑ and turn‑level objectives with the hybrid advantage and unified critic. We conduct extensive evaluations of HyGAE across five multi‑turn decision‑making environments, where it achieves an average success rate of 91% and a significant improvement of 10% over other methods. Furthermore, we provide an in‑depth analysis showing that the exact analytic form of the hybrid advantage and return is crucial for optimization. Project Page: https://wx‑zhang.github.io/hygae‑web/.
Authors:Guo Yurong, He Yufei, Li Yonghao, Chang Dongliang, Zhang Ke, Ma Zhanyu
Abstract:
Controllable infrared‑visible image fusion aims to integrate complementary thermal and structural information with flexible region‑aware modulation, producing fused images that adapt to diverse user requirements and downstream tasks. However, existing methods typically rely on predefined discrete control conditions, leading to a sparse space that fails to support fine‑grained modulation demands. To address this, we propose ConFusion, a novel framework that learns the continuous fusion space via Gaussian‑conditioned spatial‑aware modulation, enabling instance‑level fine‑grained controllable infrared and visible image fusion. ConFusion employs a dual‑branch architecture to disentangle modality‑invariant and modality‑specific representations under joint reconstruction and text‑guided semantic alignment. Gaussian‑conditioned instance modulation variables coupled with Grounded SAM‑based instance masks guide instance‑level fine‑grained modulation through the Mask‑Guided Specific Feature Modulator, while the Text‑Driven Invariant Feature Enhancer improves semantic consistency and enhances fusion. During inference, the multimodal large language model parses user intents into instance‑level modulation variables to guide image fusion. Extensive experiments show that ConFusion achieves state‑of‑the‑art performance across multiple metrics in both fusion quality and downstream tasks, while supporting fine‑grained controllable image fusion. Our code is available at https://github.com/HeyufeiAnto/Confusion
Authors:Ruiyi Yan, Yugo Murawaki, Zhongliang Yang
Abstract:
Generative linguistic steganography conceals secret bits within the sampling randomness of large language models. Existing schemes are single‑stream, conveying an entire secret through a single response to a single prompt. This convention incurs two limitations: it provides no protocol‑level support for batched multi‑stream inference, and naive co‑batching does not conceal slot occupancy or payload completion. We propose HiTMS, which distributes a secret across multiple responses produced jointly over successive rounds of interaction. Each round embeds and extracts several streams within a single batched call, thereby amortizing the cost of model invocation and substantially improving throughput. To ensure recoverability, HiTMS wraps each response in a self‑describing frame and employs a key‑derived schedule that binds streams to slots and fills unused slots with decoys, guaranteeing exact recovery while concealing the number of active streams. The framework is agnostic to both the language model and the steganographic coder. Across eight dataset‑model‑coder settings, eight‑stream HiTMS achieves up to 4.3 times higher embedding and extraction speeds than single‑stream baselines, while reducing the steganalyzer AUROC from 0.681 to 0.601 on average. Additional experiments with 4 to 64 streams demonstrate sustained throughput gains as concurrency increases. GitHub repository for this work is https://github.com/ryehr/HiTMS_steganography.
Authors:Yunlong Lin, Zixu Lin, Zhaohu Xing, Biqiang Li, Chenxin Li, Haonan Wang, Haitao Wu, Hengyu Liu, Jianghai Chen, Kaituo Feng, Kaixin Li, Shawn Chen, Shijue Huang, Sixiang Chen, Tsung-Yi Ho, Wenxuan Huang, Xiangyan Liu, Xiaomeng Hu, Xuanhua He, Yan Sun, Yunqing Zhao, Zhiqin Yang, Zehan Wang, Zhengyang Tang, Tianyu Pang, Xiangyu Yue
Abstract:
Creative AI is moving from single‑step asset generation toward long‑horizon multimodal production. Although recent generative models can synthesize high‑quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real‑world creative work requires more than isolated prompt‑output interactions. It involves references, drafts, alternatives, edits, failed attempts, version relations, tool actions, evaluation signals, and human feedback, which together form an evolving project state. Existing prompt‑based, chat‑based, and node‑based generation systems only partially support this state, as they often discard intermediate context, rely on linear conversations, or require manually specified workflows. Recent commercial systems indicate a shift toward agent‑assisted creative production, but their closed architectures make it difficult to study how agents represent context, choose tools, revise artifacts, recover from failures, and maintain consistency over time. To address this gap, we introduce JarvisHub, a canvas‑native creative agent harness for long‑horizon multimodal creation. JarvisHub treats an editable canvas as the user workspace, the agent's external memory, action space, and shared project state, representing multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links. Through a three‑layer architecture of canvas state, protocol bridge, and agent runtime, JarvisHub enables agents to act within an inspectable and editable creative state. This design moves creative agents beyond isolated tool use toward sustained, human‑steerable creative automation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.
Authors:Fabio Aurelio D'Asaro
Abstract:
FastLAS is a scalable system for Inductive Logic Programming (ILP): you give it some background knowledge, a language bias, and a set of examples, and it searches for a set of logic program rules (a hypothesis) that explains the examples. These notes are a hands‑on introduction to writing FastLAS programs. They are organised as a programmer's guide: syntax first, then a ladder of worked, numbered examples of increasing difficulty. Every self‑contained example here has been run against FastLAS 2.2.0 and shows the tool's actual output. We keep theory to the minimum needed to write correct programs; throughout, set‑off notes flag where FastLAS differs from its sibling system ILASP, and where the two learning algorithms (‑‑opl and ‑‑nopl) behave differently. The document is intended as an unofficial tutorial to FastLAS 2.2.0, not as an official language specification.
Authors:Jiwon Moon, Yerin Hwang, Kyomin Jung
Abstract:
Instruction hierarchy (IH) requires models to prioritize instructions by source, ensuring that higher‑priority instructions override lower‑priority ones. Despite its importance for safe and controllable deployment, existing evaluations have focused almost exclusively on English, leaving it unclear whether IH compliance remains stable in multilingual settings. We introduce XIH‑Bench, a benchmark for multilingual IH evaluation with both same‑language and cross‑language conflicts across six languages, four domains, and three IH settings. Across models, we find two consistent patterns. First, IH compliance exhibits a clear language‑dependent asymmetry: a language that strengthens compliance in the higher‑priority position can become disruptive in the lower‑priority position. Second, cross‑language conflicts yield higher compliance than same‑language conflicts, a phenomenon we term the Language Boundary Effect. We further show that language specialization can make lower‑priority instructions in model‑favored languages harder to override, creating multilingual reliability and security risks.
Authors:Qiao Yan, Yihan Wang, Zhenghao Xing, Jiaqi Xu, Pheng-Ann Heng
Abstract:
Autonomous driving under adverse weather remains a critical challenge, yet existing vision‑language benchmarks mainly evaluate under standard conditions, synthetic corruptions, or single modality. As a result, it remains unclear how vision‑language models behave under real‑world adverse weather with multi‑modal inputs. We argue that a key difficulty lies in degraded environmental observability: under fog, rain, snow, and low illumination, multi‑modal observations become unreliable and cross‑modally inconsistent, posing challenges to scene understanding, and subsequent decision‑making. To study this, we introduce ObsDriveBench, a real‑world multi‑modal benchmark for adverse‑weather autonomous driving. Our benchmark is designed with three capability dimensions: observability awareness, spatial reliability, and risk‑aware decision‑making, enabling fine‑grained diagnosis of model behavior under degraded observations. We construct the benchmark through observability meta‑annotation, scene description, and capability oriented multiple‑choice tasks over synchronized camera, LiDAR, and radar inputs, forming a benchmark with over 14k training and 13k test questions. Experiments reveal consistent performance degradation of existing vision‑language models. We further introduce ObsDrive model with normal‑weather supervised fine‑tuning and adverse‑weather reinforcement learning, improving robustness across all three capabilities. The dataset and evaluation code will be released at \hrefhttps://github.com/russellyq/ObsDriveBench\textttObsDriveBench.
Authors:Bartol Bućan, Nikola Sočec, Sarah Isufi, Morena Granić, Luka Hobor, Agneza Krajna, Mihael Kovac, Mario Brcic
Abstract:
Political audits of large language models (LLMs) usually reduce each to one point on a political compass. But that resting point barely matters in deployment: a model must land somewhere, and what counts is how far, and in which directions, its answers can be steered. That steering runs through the system prompt: the personalization layer a platform sets, or one induced from a user's history, not necessarily written by hand. We run a dispersion‑first stress test of prompt‑based controllability across 12 ideological personas plus an unsteered baseline, 70 Political Compass items, ten replicates, and seven leading LLMs: GPT‑5, Claude, Grok, Gemini, DeepSeek, Kimi, and Qwen (63,700 responses). Contextual framing explains roughly 88%‑93% of variance on the economic and society axes, model identity under 3%: responses are highly instruction‑adjustable. Models do not shift alike: some move more, and some saturate under extreme framings. Conflicting directional‑steering results in prior audits resolve once baselines are recognized as non‑centered: displacement and proximity diverge, so the effect is geometric, not differential compliance. Under authoritarian prompts, models produce similar shifts on the same questions. Political‑coordinate audits therefore need steerability audits reporting dispersion, symmetry, saturation, and refusal floors. We release prompts, benchmark data, and code.
Authors:Hengyuan Cao, Shizhuo Cheng, Mingxuan Liu, Weicheng Huang, Yunhong Lu, Chenxi Cai, Yan Zhang, Min Zhang
Abstract:
The rapid evolution of generative models has unlocked new potentials in protein binder design, a pivotal task in structural biology, by facilitating end‑to‑end generation via joint sequence‑structure modeling or hallucination. However, existing approaches are predominantly implemented under a single‑target, single‑state assumption, limiting their ability to model multi‑target or multi‑state interactions required for advanced function‑oriented protein design. Here, we introduce Chamaileon, which unifies multi‑target and multi‑state binder design by formulating the problem as cross‑context binding landscape modeling. The framework is underpinned by a training paradigm termed In‑Context Complex Co‑Design (I3CD) for context‑aware sequence‑structure co‑modeling. During inference, we employ Mixture‑of‑Paths Sampling (MoPS), a scalable strategy that optimizes a single sequence across contexts while alleviating the scarcity of high‑quality multi‑conformational paired data. Extensive evaluation on our newly constructed benchmark, CROSS, demonstrates that Chamaileon effectively generates sequences adaptable to diverse conformational landscapes and multi‑target requirements. The code is available on https://github.com/caohengyuan/Chamaileon.
Authors:Zhijing Cheng, Xuancheng Zhang, Donglin Di, Lei Fan, Baorui Ma, Hao Li, Xun Yang
Abstract:
End‑to‑end autonomous driving systems commonly follow a cascaded two‑stage pipeline where a perception stage compresses multi‑modal sensor inputs into a compact context and a downstream planner predicts trajectories conditioned on this context. We argue that this one‑way perception‑to‑planning interface forces sensor inputs into a compact representation, losing the fine‑grained details critical for planning. Moreover, by constraining the planner to this compressed context, it is difficult to leverage the rich representations offered by modern vision foundation models. To address these issues, we propose MOJITO, a unified sensor‑to‑action framework for end‑to‑end autonomous driving built on modal joint learning. MOJITO removes the cascaded interface and instead performs block‑wise Modal Joint Attention that simultaneously updates action, image, and LiDAR features, allowing the planner to directly access multi‑modal features during action generation. MOJITO achieves 88.9 PDMS on the NAVSIM v1 dataset and 88.4 EPDMS on the more challenging NAVSIM v2 dataset, setting a new state‑of‑the‑art. Extensive experiments further demonstrate strong scalability, instruction following, and diverse trajectory generation. Code and models are available at https://github.com/mumucc01/MOJITO.
Authors:Gyeongwon Jeong, Seonghun Park, Jihoon Hyun, Sang-il Oum, Hongseok Yang
Abstract:
Razborov's flag algebra method is a powerful tool for proving asymptotic inequalities in extremal graph theory, often reducing the task to finding a finite certificate by semidefinite programming. We present a machine‑checked formalization of the method for finite simple graphs, together with a certificate‑to‑proof compiler that turns externally generated certificate data into algebraic proofs checked by Lean. The formalization covers the foundations of the method: partially labeled graphs, their densities in large graphs, the quotient algebra of density expressions, graph‑limit semantics through positive homomorphisms, and the downward operators used to average out labels. The compiler treats the external semidefinite programming output as candidate data rather than trusted input: Lean independently computes the required density and multiplication facts, verifies positive semidefiniteness exactly over \mathbbQ, and carries out the algebraic normalization steps of flag‑algebra proofs. Our case studies yield formal proofs of seven Turán‑type upper bounds, including Mantel's theorem and the Erdős pentagon theorem, a C_4‑density bound for triangle‑free graphs, and edge‑density bounds for K_4‑free, K_5‑free, and C_5‑free graphs. Independently of the compiler, we formalize the matching constructions that complete the exact Turán densities of Mantel's theorem and the Erdős pentagon theorem, and prove two inequalities of Goodman. Our constrained semantics also prompted a meta‑theoretic comparison of two ways of imposing graph constraints: building a hereditary constraint into the flag algebra from the start, or testing inequalities afterward on constrained graph limits with labels chosen at random. We state the resulting root‑plantability criterion characterizing when the two approaches agree; a forthcoming paper will present the complete account.
Authors:Jianhang Xie, Sicheng Tan, Vishnu Naresh Boddeti, Zhichao Lu
Abstract:
Fully homomorphic encryption (FHE) provides strong cryptographic guarantees for private inference, but deploying transformer models under FHE remains prohibitively expensive. A key bottleneck is that non‑linear operations such as softmax, normalization, and activation must be replaced with polynomial approximations compatible with the CKKS scheme, and the multiplicative depth consumed by these approximations dominates inference cost. Recent frameworks have advanced approximation techniques, yet all rely on manually configured approximation hyperparameters (e.g., number of iterations, polynomial degree), applied uniformly across all layers. While convenient, this uniform‑configuration approach is overly rigid: different layers can tolerate different levels of approximation error without degrading predictive accuracy, and uniform configurations cannot exploit this variability to reduce latency. Allowing each layer to adopt its own configuration, however, causes the search space to explode with model depth, reaching roughly 10^84 configurations for BERT/ViT (12 layers) and 10^225 for LLaMA3 (32 layers), rendering manual exploration practically impossible. We present ATLAS, an automated framework that configures per‑layer approximation settings by formulating the problem as a multi‑objective optimization over latency and predictive accuracy. The resulting problem is inherently difficult: 1) competing objectives over a large decision space (120 or 320 variables for BERT/ViT or LLaMA3); 2) expensive evaluation, as each configuration takes 70‑1,000 seconds even in cleartext; and 3) sparse optimization signals, as 35‑50% of candidate configurations yield numerically invalid solutions. ATLAS addresses these challenges through a two‑stage optimization strategy that progressively relaxes layer‑wise constraints, combined with surrogate models to accelerate evaluation.
Authors:Seung Hyun Lee, Stella X. Yu
Abstract:
Robot policies are typically MLPs mapping observations to actions. Yet robot observations are physical variables, and many action‑relevant cues arise not from individual variables but from their interactions; power, inertial effects, contact, slip, and compliance depend on products among observable signals. We introduce PRISM, a policy representation that makes polynomial interactions among observable physical variables explicit, learnable, and compact. Rather than listing all polynomial terms, PRISM uses a factorized polynomial module to expose higher‑order interaction features efficiently. In reinforcement learning, it keeps the standard MLP backbone but applies a gradually activated element‑wise polynomial function after it. In imitation learning, it replaces linear proprioceptive conditioning in Diffusion Policy with a polynomial layer trained end‑to‑end. Across humanoid locomotion and contact‑rich manipulation, PRISM improves performance over standard MLP policies and larger MLPs with matched capacity, showing that interaction structure cannot be replaced by capacity alone. It also yields sensorless compliant behavior without force, wrench, tactile input, contact labels, or admittance control. These results suggest that polynomial representations should become a standard architectural choice for embodied motor control. The project page is available at https://lsh3163.github.io/prism/
Authors:Longying Wen, Feiyang Wu, Jinglin Yu, Chongxian Yuan, Renjie Li, Zhaoyu Zhang
Abstract:
Photonic‑crystal surface‑emitting lasers (PCSELs) can combine high‑power operation with narrow‑divergence surface emission, but optimizing coupled parameters requires costly full‑wave simulations. Deep Q‑network (DQN) optimization can reuse simulated transitions to guide edits, yet which value‑learning mechanisms remain reliable under tight simulation budgets is unknown. We address this gap by comparing baseline DQN and six value‑based variants for a seven‑variable PCSEL design under a shared objective, simulator, 83‑call budget, and four matched initializations. Beyond endpoints, we analyze sample efficiency, policy behavior, and physical response to separate learning gains from favorable starts or exploratory jumps. Dueling DQN is the only variant to improve all four seeds. Relative to the first evaluated designs, its selected structures increase the mean quality factor () from to (), reduce wavelength error by 64%, and increase upward power by 47%; compared with baseline DQN, they achieve a higher mean under the same budget. Other variants yield no consistent improvement; Double DQN reproduces baseline trajectories, while Rainbow‑lite shows high upside but strong seed dependence. These results identify Dueling DQN as the most reliable configuration tested for simulation‑budget‑limited PCSEL inverse design and provide a reproducible framework for attributing algorithmic gains in scientific optimization. The source code is publicly available at https://github.com/Longying‑Wen/PCSEL‑RL.
Authors:Weixiang Zhou, Jiabei Zuo, Yuhao Wang, Cong Wang, Huchuan Lu, Zhixun Su
Abstract:
Multi‑modal object Re‑Identification (ReID) aims to retrieve specific objects by integrating complementary information from multiple modalities. However, existing multi‑modal ReID methods do not effectively address background interference suppression or achieve tri‑modal alignment, instead focusing on pairwise feature fusion. Moreover, many current aggregation approaches suffer from high computational complexity. To address these limitations, we propose PRISM, a novel multi‑modal ReID framework built upon Prompt‑S6 (PS6) and semantic‑aware knowledge guidance. PS6 maintains the linear complexity and strong sequence modeling capability of Mamba while enabling efficient cross‑modal interaction. Leveraging these advantages, we design two key components: Semantic‑Driven Token Pruning (SDTP) and Progressive Fusion Network (PFN). Parsing semantic priors from the segmentation foundation models, the SDTP then leverages these priors and applies dynamic token pruning to suppress background noise and refine feature representations. The PFN progressively aggregates multi‑modal features to achieve tri‑modal alignment and fully exploit modality complementarity. With the proposed modules, PRISM generates more robust multi‑modal representations under complex scenarios. Extensive experiments on four multi‑modal object ReID benchmarks demonstrate the effectiveness and efficiency of our approach. The source code is available at https://github.com/zw‑absin/PRISM.
Authors:Yiming Zhong, Chang Nie, Caifeng Shan
Abstract:
Omnimodal large language models (OmniLLMs) are rapidly extending multimodal reasoning to cover synchronized audio and video. However, the resulting audio‑video token sequences are long, leading to high prefill latency and GPU memory usage at inference time. Existing token pruning methods, designed mainly for vision‑only inputs, miss both the cross‑modal links between audio and video and the user query that decides which content matters. To bridge this gap, we present Omni‑Prune, a training‑free, query‑aware audio‑visual token pruning framework that jointly removes redundancy from both modalities while keeping task‑relevant cross‑modal evidence. Specifically, Omni‑Prune first splits the token sequence into adaptive time windows placed at audio saliency peaks, then scores audio and video tokens on a single scale that combines encoder attention with text‑query relevance, and pairs related audio‑video tokens so that they are kept together. Within each window, a final K‑medoids step then selects a few representative tokens, adding diverse cues that score‑based selection alone would miss. Extensive experiments demonstrate that Omni‑Prune outperforms established baseline methods, delivering up to 3.25x prefill speedup and 1.3x memory reduction while retaining over 99% of full‑model performance.
Authors:Adhyyan Narang, Artin Tajdini, Claire Zhang, Jamie Morgenstern
Abstract:
Recent work shows that fine‑tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data can transmit hidden preferences that generalize broadly. Standard defenses, such as data filtering, mixing in harmless data, and regularization, attenuate these effects but do not eliminate them. We instead pursue robustness through redundancy: collecting multiple datasets from different sources and only learning what is common between them. Thus, if only a subset of sources are malicious, the misbehavior will be blocked. In order to implement this defense strategy, we fine‑tune a separate reference model on each source's dataset and aggregate their next‑token distributions at decoding time. We introduce two consensus decoders: a token‑wise minimum, which caps each token at the lowest probability any source assigns, and a base‑relative variant, which reverts to the base probability on any token the sources move in opposing directions. We further relax exact agreement to tolerate partial support across sources and different surface expressions of the same intention. Across controlled poisoning tasks, subliminal learning, and emergent misalignment, consensus decoding suppresses source‑specific misbehavior while preserving shared desirable behavior, including cases where union training and weight averaging retain the unwanted behavior.
Authors:Paul Simpson, John Kozak, Lisa Doake
Abstract:
We document a failure class in frontier large language models ‑‑ exception chain collapse ‑‑ observed in eligibility evaluation under nested conditional rules of the form "A is required UNLESS B applies, UNLESS C overrides B". The failure reproduces at first observation, but its empirical surface is unstable: between March and April 2026 several failure cells closed silently under the same model alias, with no version bump (GPT‑5.4 on construction insurance moved from 96.6% to 100%, same prompt and harness). For regulated workflows, frontier‑model accuracy is a moving compliance boundary that shifts without notice. We present the Aethis Eligibility Module, a neuro‑symbolic architecture in which LLMs author rules from authoritative sources and an SMT‑based layer executes them deterministically, consistent with the authored specification regardless of model drift, reasoning‑effort defaults, or prompt format. Three evidence bases: (i) a controlled benchmark of 225 scenarios across four regulatory domains documents the pattern and, in replication, the drift that partially closed it; (ii) a 20‑scenario adversarial extension on construction insurance, where the engine scores 20/20, as does one of four frontier configurations (GPT‑5.4 at low reasoning effort), while the other three, including Anthropic's strongest model at evaluation time, fail the same coverage‑gap edge case; (iii) external validation on nine peer‑reviewed LegalBench tasks, 949 held‑out cases, where the engine is significantly more accurate than all three frontier models (combined McNemar's p <= 0.003), with margins up to +41 points on the curated multi‑prong tasks against the Anthropic models. The contribution is to relocate uncertainty from the inference boundary, where it is silent, to the specification boundary, where it is deliberate and audited. All scenarios, rule encodings, and results are public and reproducible.
Authors:Aditya Dewan, Arjun Yogeswaran, Benjamin Fedoruk
Abstract:
Modern deep neural networks are potent catalysts for scientific and industrial impact, yet excessive parameter counts impede deployment in low‑compute settings such as hospital equipment and energy infrastructure. Predominant knowledge distillation (KD) methods favor replication: smaller students mimic teacher output logits, yet empirically yield low task performance, hamper convergence, and act merely as regularization rather than substantive knowledge transfer. We propose Saddle Point Recruitment for Knowledge Distillation (SPRKD), reframing distillation from replication to employing teachers as optimization‑curvature and domain proxies, characterizing saddle points as regions of strong further‑descent potential via embedding and basin‑fractal properties. Using Hessian eigenvalue spectral density (ESD), SPRKD identifies low‑loss saddle regions for student re‑exploration; weak‑teacher ensembles are aggregated into an Approximated Saddle Region (ASR), re‑parameterized into the student via Transfer Learning by Injection, and approached with exponentially decaying Euclidean transformations, Negative Hessian Eigensteps, and Gaussian perturbations. On malaria blood smear classification with a 6,430‑parameter CNN distilled from a weak 25,546‑parameter teacher, SPRKD reaches 94.8% validation accuracy, outperforming Response KD by 24.70 percentage points (McNemar p = 6.3e‑87) and matching scratch‑trained baselines of the same architecture to statistical equivalence (p = 1.0). Across MNIST, CIFAR‑100, and TinyImageNet, SPRKD exceeds scratch‑trained baselines by up to 8 percentage points on preliminary benchmarks. Hessian ESD and 2‑D loss landscape analysis show convergence to wider minima with substantially smaller Hessian trace and spectral radius than Response KD and control students, indicating smoother descent and greater noise robustness.
Authors:Giovanni Sullutrone, Luca Sala, Sania Aftar, Georgia Koutrika, Sonia Bergamaschi
Abstract:
Large Language Models (LLMs) demonstrate high performance on curated Text‑to‑SQL benchmarks; nevertheless, real‑world users frequently pose ambiguous or unanswerable questions that current systems handle poorly. Three interconnected gaps hinder progress: incomplete taxonomies, realistic benchmark generation for real‑world settings, and static user interaction. We address all of the above issues through three contributions: (1) a unified taxonomy of 8 categories covering ambiguous and unanswerable questions; (2) a multi‑agent generation pipeline with a two‑stage process (NLQ generation followed by SQL grounding) and an explicit Category Conformance validation stage, producing questions from arbitrary databases validated by a council of local open‑source models; and (3) ABISS (Ambiguity Benchmark using Interaction‑Simulated Sessions), a dynamic simulation environment where Text‑to‑SQL agents interact with style‑aware simulated users across multi‑turn dialogues. Experiments with eight open‑source models on ABISS‑BIRD and ABISS‑Spider reveal two fundamental bottlenecks. The first is subcategory classification: models detect that a question is problematic yet struggle to pinpoint the specific subcategory. The second is clarification‑conditioned SQL generation: even after receiving useful user information, models often still fail in the final resolution step. Providing the ground truth category yields large gains in both execution and feedback across both datasets, yet ambiguous‑question execution remains low even under oracle category labels. We release our code for data generation and benchmark on GitHub (https://github.com/giosullutrone/ABISS‑Evaluating‑Text‑to‑SQL‑Systems‑Through‑Agent‑Interaction).
Authors:Kawshik Banerjee, Khaled Mohammed Saifuddin
Abstract:
Graph compression reduces the computational cost of graph learning, but its effect on signal propagation remains largely underexplored. Existing work evaluates compression through downstream task performance or structural preservation, neither of which directly captures how propagation dynamics change after compression. We study two fundamental compression paradigms, coarsening and sparsification, and ask whether they preserve the propagation behavior of the original graph. Across five datasets, varying compression rates, and propagation depths, we measure signal behavior through three complementary metrics. Our results reveal a consistent tension between the two compression families. Sparsification retains higher signal diversity and mitigates oversmoothing, but its propagation trajectory progressively diverges from that of the original graph. Coarsening more faithfully preserves propagation behavior, but at the cost of stronger smoothing and rank collapse. These findings demonstrate that two propagation‑centric objectives, preserving signal diversity and preserving propagation fidelity, are distinct and empirically at odds under graph compression, highlighting the need for evaluation protocols that jointly consider both dimensions. The code and results are available at: https://github.com/KawshikBanerjee/Compression‑Propagation‑Duality
Authors:Yue Yao, Caleb N. Ellington, Jingyun Jia, Baiheng Chen, Dong Liu, Rikhil Rao, Jiaqi Wang, Samuel Wales-McGrath, Yixin Yang, Zhiyuan Li, Eric P. Xing, Ben Lengerich
Abstract:
Modern predictive systems are expected to adapt their behavior to the specific situation they are facing. A clinical model should not treat every patient the same; a retrieval‑augmented model should change its answer when given different evidence; a mixture‑of‑experts model should route different inputs to different experts. We call this capability context‑adaptive inference: before predicting, the system uses information about the current context to specialize its parameters or computation for that instance. This article provides a unified view of context‑adaptive inference across three traditions that are usually treated separately: (i) explicit adaptation in statistics (e.g. varying‑coefficient models, local regression, hierarchical sharing), (ii) rapid task‑specific adaptation in meta‑learning and transfer, and (iii) implicit adaptation in large foundation models via prompting, retrieval, and expert routing. We formalize these approaches under a common objective: to map context c to adapted parameters θ(c), then to predict via f(x; θ(c)). Under squared loss, linear prediction heads, and fixed features, we prove that explicit parameter adaptation and implicit routing are mathematically equivalent to kernel ridge regression on joint features of inputs and context. Building on this bridge, we propose practical design principles and evaluation metrics including adaptation‑efficiency, routing stability, and context‑specific robustness to guide when to specialize, how to constrain that specialization, and how to audit context‑adaptive models in deployment. Finally, we identify open problems in identifiability, robustness under distribution shift, and efficient large‑scale adaptation, outlining design principles for methods that are scalable, reliable, and transparent in real‑world settings.
Authors:Shaoheng Xu, Chunyi Sun, Jihui Zhang, Amy Bastine, Prasanga N. Samarasinghe, Thushara D. Abhayapala
Abstract:
Image‑source‑method (ISM)‑based room impulse response (RIR) simulation is a useful and physically interpretable tool for acoustic scene modeling, but full‑order ISM becomes computationally expensive as the reflection order and room complexity increase. We propose a physics‑guided framework for fast RIR simulation that preserves the geometric structure of ISM while learning to retain only acoustically important image‑source paths during online traversal. To recover energy removed by pruning, the proposed PathRIR uses a lightweight compensation multilayer perceptron to predict the missing late‑tail energy envelope and generate a compensation tail whose energy follows that envelope. Experiments on irregular 3D rooms show that PathRIR reduces image‑source computation and improves runtime efficiency over a full‑order ISM simulator, while achieving low waveform‑ and decay‑related errors. Ablation results show that adding the compensation tail improves waveform fidelity and reduces energy‑decay‑curve error, reverberation‑time error, and direct‑to‑reverberant‑ratio error, with modest runtime overhead.
Authors:Chen-Yi Lu, Yueh-Shao Chen, Somali Chaterji
Abstract:
Contrastive vision‑language models such as CLIP map semantically opposite phrases (e.g., "a dog" vs. "not a dog") to nearly identical embeddings, rendering them insensitive to negation. We attribute this failure to a phenomenon we call Representational Collapse: by tracking compositional divergence and visual alignment across the CLIP text encoder, we show that middle layers build compositional syntax, but the final layers collapse this structure as visual alignment rises, producing a syntax‑blind final representation. To recover the lost negation signal without altering pretrained weights, we propose PeakPatch, a lightweight post‑hoc correction system that intercepts the encoder at its compositional peak. An Embedding Correction Network (ECN) uses cross‑attention to extract a negation‑specific signal from the peak layer, anchored to a stable baseline, and predicts a deviation vector that re‑injects the lost syntax into the final‑layer embedding space. A complementary Score Correction Network (SCN) predicts bounded scalar score offsets for discriminative tasks. Both modules are trained jointly end‑to‑end while all CLIP parameters remain frozen, adding only 5.2M parameters (3.5% of the backbone) and preserving the standard cosine similarity interface. On NegBench, PeakPatch achieves 74.3% on COCO MCQ (+35.1 over CLIP, +17.8 over the best encoder fine‑tuning method) and 65.5% on VOC MCQ, while outperforming all fine‑tuning baselines on fully out‑of‑distribution negation retrieval despite training only 3.5% of the parameters. The corrected embeddings also transfer to text‑to‑image generation (+18.4 negation score) and generalize across ViT‑B/32, ViT‑L/14, and SigLIP backbones. Project URL: https://stevencylu.github.io/PeakPatch/.
Authors:Zhijiang Tang, Jiaxin Qi, Kaihua Tang, Yuhua Zheng, Jianqiang Huang
Abstract:
Image captioning is a primary task in vision‑‑language research, yet assessing how faithfully a caption preserves image semantics without relying on reference captions remains unsettled. Prevailing evaluations rely on human‑annotated references, whose content reflects annotator intent and captioning proficiency. In this paper, we study a reconstruction‑based principle for caption evaluation: a caption is as good as its capacity to enable reconstruction of the original image. However, because captioning inherently compresses visual information, it is impossible to recover all details, and pixel‑wise comparison between reconstructed and source images is neither feasible nor meaningful. Through our in‑depth analysis of the nature of captions, whose fundamental purpose is to transmit the semantic content of an image, we propose a revised principle: a caption is as good as its capacity to enable a reconstruction that is semantically equivalent to the original. To assess semantic equivalence, we test whether the reconstruction matches the original image across a suite of downstream vision‑‑language tasks, yielding a reference‑free, task‑conditioned caption score. We characterize component‑dependent limitations and introduce the lower‑cost Captioning Turing Test Dataset (CTTD) surrogate.
Authors:Jiajun Zou, Jiawei Liu, Ao Liu, Junnong Tian, Yibin Zhang, Chengjie Liu, Yuxi Wang, Shan Shen, Wenhua Gu, Jun Yang, Wenjian Yu
Abstract:
As chip manufacturing processes advance to deep submicron nodes, parasitic interconnect effects increasingly dominate the performance of analog and mixed‑signal (AMS) circuits and often lead to costly layout iterations. This makes early‑stage estimation of parasitic capacitance and resistance important for parasitic‑aware design exploration before full physical implementation. However, progress on GNN‑based parasitic modeling has been hindered by the lack of public, high‑fidelity RC benchmarks that support reproducible evaluation. To address this gap, we introduce ParasGB, the first open‑source benchmark suite for pre‑layout parasitic parameter prediction on circuit graphs. ParasGB provides large‑scale, heterogeneous RC networks extracted with commercial EDA tools from tape‑out‑proven designs, together with a unified evaluation protocol covering node‑level ground capacitance, edge‑level resistance, and edge‑level coupling capacitance. Within this framework, we benchmark diverse GNN architectures using a standardized training pipeline and expose challenges such as extreme label imbalance, long‑tailed parasitic distributions, and strong structural heterogeneity. By establishing a physically grounded and standardized benchmark for early‑stage parasitic prediction, ParasGB provides an open platform for reproducible research on circuit graph learning and parasitic‑aware model development. All datasets, preprocessing scripts, and configurations are publicly available in our code repository https://github.com/ShenShan123/ParasGB.git.
Authors:Juan Manuel Castillo Pinto
Abstract:
We present BoneAgeTW2, the first fully open‑source system to automate the complete Tanner‑Whitehouse 2 (TW2) clinical protocol for skeletal maturity assessment end‑to‑end. The system employs YOLOv8 for precise detection and localization of the 20 TW2 hand bones from radiographic images, and an EfficientNet‑B3 backbone with 20 independent classification heads to assign maturation stages (A‑I) to each bone simultaneously. From these predictions, the system automatically generates clinical PDF reports including interactive Gaussian distribution curves for all 20 bones, enabling direct comparison with population norms. The model is trained on the public RSNA Pediatric Bone Age Challenge dataset (12,611 hand radiographs) using a pseudo‑labeling strategy to derive per‑bone stage labels from global bone age annotations. The full codebase is publicly available at https://github.com/jmmana/BoneAgeTW2.
Authors:Zobeir Raisi, John Zelek
Abstract:
Scene Text Recognition (STR) models are trained almost exclusively on word crops of at most 25 characters, yet real deployments (signage, product labels, dense captions) require reading much longer text. This paper diagnoses that failure and then closes it. The diagnosis separates out‑of‑length failure into two simultaneously extrapolating axes (the encoder's width axis and the decoder's time axis) and shows that encoder width, not decoder length, is the dominant failure mode. Representation‑side fixes bring only partial relief: training‑free rotary rescalings recover at most 2‑4 points of character error rate (CER), and a weighted fine‑tuning recipe recovers 6‑8 points while improving standard‑benchmark accuracy, yet word accuracy on the Long Text Benchmark (LTB) stays near zero, because the residual gap lies in the decoding mechanism rather than the representation. We then close that gap at inference time, on an unmodified word‑level checkpoint: the long image is sliced into overlapping crops at the model's training width, each decoded independently and in‑distribution, and the reads stitched by geometry‑anchored edit‑distance alignment. This procedure reaches 42.79‑43.05% bucket‑average word accuracy on LTB across two base checkpoints, matching the published state of the art (41.57%) and beating it by 11‑12 points on the hardest bucket, at wall‑clock parity with plain decoding; applied unchanged to the public PARSeq checkpoint it reaches 47.11%. Once chunking is applied fine‑tuning no longer helps: the decoding‑side fix alone matches purpose‑built architectures. We release the diagnosis harness and implementation.
Authors:Jinsen Su, Yongdong Luo, Yuexiao Ma, Yibo Hu, Meiguang Jin, Xiaowu Zheng
Abstract:
Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross‑modal salience mismatch makes unidirectional guidance prone to discarding answer‑critical cues under aggressive compression. We propose OmniScope, a training‑free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality‑specific token budgets, prunes visual tokens with an anchor‑delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio‑video benchmarks and two Qwen2.5‑Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35‑point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC‑AutoML/OmniScope.
Authors:Summer Sun
Abstract:
Existing evaluations of large language models cover knowledge, reasoning, coding, and tool use, but they rarely treat a verifiable deliverable produced within a constrained workflow as the unit of evaluation. We introduce SQBench, a benchmark for evaluating production‑oriented task delivery by language‑model agents. SQBench v1.0 contains 220 standardized tasks organized into L1 atomic capabilities, L2 composite skills, and L3 business scenarios. Each task requires an agent to process input assets, use available tools, and produce an explicitly specified deliverable. The evaluation first computes functional Completion and then derives Risk Penalty and Performance from independently evidenced triggers in a 10D Risk Matrix. A Strict Pass requires Completion = 1 and Risk Penalty = 0. We evaluate 27 model configurations under a common protocol, with one run per configuration‑task pair. The highest prespecified Weighted Pass@1 is 60.5%. Mean Strict Pass@1 on L3 is 18.5%, and every configuration performs worse on L3 than on both L1 and L2, indicating that delivery under domain constraints is a shared weakness within the current task set. Of 2,348 results with Completion = 1, 113 (4.8%) fail the Strict Pass criterion because of risks such as unverifiable citations, inappropriate resource use, or format violations. These results show that functional completion alone does not fully characterize delivery quality and that risk determinations should be reported separately.
Authors:Lukas Förner, Melina Wördehoff, Julian Steffens, Maximilian Schmutz, Rainer Claus, Josua Decker, Thomas Kröncke, Kartikay Tehlan, Thomas Wendler
Abstract:
Longitudinal medical imaging captures temporal evolution of lesions, yet extracting the underlying dynamical parameters governing this evolution remains challenging. We propose an inverse Bayesian framework for inferring lesion dynamics from longitudinal spectral CT. We decompose spectral feature (x) evolution into three components: \beginequation \fracdx_idt = A_i x_i + B \cdot n + C \cdot Δx_\textsat \endequation where A_i captures intrinsic dynamics (lesion‑autonomous evolution), B captures local environment tumour burden (organ tumour burden through satellite count coupling), and C captures environment/satellite state change (i.e., whether surrounding lesions move similarly or not). We demonstrate the framework on photon‑counting NSCLC CT data from metastases, recovering distinct dynamical regimes: lung lesions exhibit significant satellite count coupling (B=‑0.34, p<0.05) suggesting competitive dynamics, while liver lesions show synergistic satellite behaviour coupling (C\approx+1.0, p<0.05). Synthetic validation confirms parameter recovery, and cross‑coupling analysis validates that our method detects non‑zero coupling when present. This work establishes inverse dynamical inference as a principled methodology for extracting interpretable parameters from longitudinal imaging, moving beyond static feature extraction toward mechanistic characterisation of lesion behaviour. The code and data are available at: https://github.com/lukasf98/inverse‑bayesian‑inference
Authors:Christian Bongiorno, Efstratios Manolakis, Rosario Nunzio Mantegna
Abstract:
This paper introduces a compact reformulation of a modular end‑to‑end neural network for global minimum‑variance portfolio optimization that decouples model complexity from both look‑back window length and universe size. A five‑parameter hyperbolic weighted moving average combined with a saturating exponential replaces the original 2,400‑parameter lag‑transformation layer, and a bidirectional gated‑recurrent‑unit eigencleaning module together with a streamlined marginal‑volatility network reduce total learnable parameters from 39,586 to just 2,175. In out‑of‑sample tests against state‑of‑the‑art nonlinear‑shrinkage and risk‑parity benchmarks, the compact network attains the lowest realized portfolio variance without compromising expected return. Under long‑only constraints, the variance reduction supports substantially higher leverage while maintaining comparable drawdown control. Validation in a high‑fidelity trading simulator that incorporates realistic margin‑call dynamics confirms enhanced over‑leverage resilience. These findings demonstrate that end‑to‑end variance‑minimization architectures can achieve substantial parameter efficiency and robust capital‑efficiency gains without sacrificing risk‑adjusted performance.
Authors:Sultan Alshehri, Zhantao Yang, Han Zhang, Marios Savvides
Abstract:
Dual‑encoder vision‑language models (VLMs) expose a similarity interface that enables zero‑shot retrieval but fails compositional constraints: queries like "umbrella and no person" retrieve images containing both, even when concept detection is reliable. We trace this to an interface‑level Bag‑of‑Concepts effect, where similarity scores approximate mean pooling of concept evidence regardless of operators. Although operator‑dependent signals exist in text embeddings, they are too weak or misaligned to affect rankings. Fine‑tuning does not reliably resolve this failure because the dominant bottleneck is how similarity aggregates evidence rather than what encoders represent. We propose factored inference, which separates evidence extraction from constraint execution, and introduce LCSE (Logic‑Constrained Score Editing), a training‑free method that executes constraints externally using concept scores from frozen encoders. We also introduce FACTOR‑Bench, where LCSE achieves 85.5% accuracy versus 73.2% for the best fine‑tuned baseline, 90.7% when applied to SigLIP 2, and improves NegBench COCO MCQ accuracy from 27.2% to 65.2% while preserving retrieval performance.
Authors:Jeff Otterson
Abstract:
Large language models increasingly write both code and the tests meant to check it; coverage records what ran, not what was verified. We study an adversarial test‑hardening loop under a mechanical oracle: a Tester model writes tests, mutation testing names surviving injected defects, and a Critic model writes tests to kill exactly those, with every verdict decided mechanically, so no model judges another's output. In Experiment 1, on five Python subjects (one same‑lineage‑loop cell could not be scored), the loop killed 105 mutants that one‑shot generation missed and lost none, and the cross‑lineage‑Critic question returned a pre‑declared null. The central finding was an autopsy: an earlier analysis reported a cross‑lineage effect at p = 9.5e‑66 that was an instrument artifact, an output cap silently truncating the verbose model, caught only by adversarial review of the completed analysis. Review then found a further confound, each arm resampling its own initial suite; Experiment 2 removes it. Under a pre‑registered frozen‑shared‑round‑0 design (five replicates on each of four subjects, seeds committed in advance), same‑lineage Critic rounds killed 78% of the survivors the frozen initial suite left standing (mean incremental kill rate 0.783, 95% cluster‑bootstrap interval [0.592, 0.935]), a within‑replicate causal estimate; the cross‑provider configuration showed a positive pilot difference (rate gap 0.178, 95% interval [0.039, 0.347]; magnitude dominated by a single replicate) at 5.5x lower arm cost. This compares two named model‑provider‑harness configurations, not an isolated lineage effect: part of the gap is one configuration's receipted operational failures, including truncation recurrences, now detected and scored rather than laundered. Cross‑model comparisons can inherit the asymmetries of the harness that runs them. We release both protocols, all receipts, and the analysis code.
Authors:Tao Zhang, Qixuan Fan, Yiyuan Liang, Yanjie Wang, Song Yan, Tian Tian, Jiahuan Zhou, Luxin Yan, Sheng Zhong, Xu Zou
Abstract:
Class‑incremental learning (CIL) requires models to continuously acquire new knowledge while avoiding catastrophic forgetting. While exemplar replay is effective, it raises concerns regarding privacy and storage. Thus, generative replay has emerged as a viable alternative, synthesizing old data using frozen pretrained text‑to‑image (T2I) models without any extra training. However, we observe that directly mixing synthetic old‑class data with real new‑class data during incremental training leads to significant performance degradation. This issue stems from a "domain shortcut", where models rely on domain‑discriminative features instead of semantic class cues. To address this, we propose DREAM (\underline\mathbfDomain‑\underline\mathbfRegularized \underline\mathbfExemplar‑free \underline\mathbfAlignment \underline\mathbfModel), which uses a training‑free generator to synthesize old‑class data and eliminates domain shortcut via subspace rectification and orthogonal projection, while reinforcing semantic alignment through real‑anchored prototype regularization. Extensive experiments on 4 datasets demonstrate that DREAM outperforms existing exemplar‑free CIL methods and achieves state‑of‑the‑art performance. Our source code is available at https://github.com/Light‑ZhangTao/DREAM.
Authors:Dhiraj Neupane, Mohamed Reda Bouadjenek, Richard Dazeley, Sunil Aryal
Abstract:
Machinery fault detection (MFD) remains heavily reliant on supervised learning, which struggles with the scarcity of fault labels in real‑world settings. While reinforcement learning (RL) offers a framework to model the sequential nature of degradation, current ``RL‑based'' MFD methods reduce the problem to a static contextual bandit (CB) formulation: by ignoring state transitions and discarding the temporal discount factor, they collapse to standard supervised classification. We propose an adversarial inverse reinforcement learning (AIRL) framework that treats MFD as an offline IRL problem. Unlike reconstruction‑based approaches that rely on static error margins, or CBs that ignore dynamics, our method recovers an intrinsic "health" reward directly from observational state transitions, requiring neither manual reward engineering nor fault labels. On three run‑to‑failure benchmarks (HUMS2023, IMS, XJTU‑SY), AIRL is the only method achieving non‑saturated post‑detection consistency across all datasets, while CB baselines fail to detect gradual degradation and reconstruction models collapse into always‑anomalous states. Code and data: https://github.com/dhirajneupane/AIRL‑MFD‑DN.
Authors:Cheng Guo, Qiming Cao, Shengkai Xu, Haoyu Xie, Kaixiang Su, Pu Wang, Hongfei Xue
Abstract:
Millimeter‑wave (mmWave) radar offers privacy‑preserving and lighting‑robust sensing for human motion reconstruction, but learning models that generalize across real deployments require diverse paired radar‑motion data that are costly to collect. Simulation provides scalable supervision, yet models trained on clean synthetic signals transfer poorly because of multipath, clutter, response statistics, and resolution degradation. We present mmSimPrior, a simulation‑pretrained framework that factorizes transferable knowledge into signal, motion, and radar‑to‑motion mapping priors. A multi‑modal signal encoder is pretrained with a physics‑informed domain‑randomization curriculum that emulates propagation‑ and acquisition‑level variations, while a joint‑temporal tokenizer learns a discrete prior over plausible human motion. A shared mapping prior supports classification over a learned motion codebook for constrained zero‑shot reconstruction and continuous regression for flexible limited‑data adaptation. We further construct a 4.2M‑frame, 31K‑sequence dataset suite and introduce a No‑Overlap Setting that excludes repeated complete subject‑environment‑location‑motion configurations across adaptation and test. Experiments on mmSimPrior‑Real and RT‑Pose demonstrate consistent gains: with only 24 paired real sequences, mmSimPrior‑Reg reduces MPJPE by 24.7% to 39.0% over the strongest baseline across the three environments, while mmSimPrior‑Cls reduces zero‑shot MPJPE by 8.5% without finetuning.
Authors:Ayush Dwivedi, Ashvi Soni
Abstract:
Few‑shot prompting, the practice of prepending a small number of input‑output demonstration pairs to a query before presenting it to a large language model (LLM), is among the most widely adopted inference‑time techniques in NLP. Yet little systematic work investigates how shot count interacts with model scale, architecture, and output format compliance in determining classification performance. This paper presents a controlled study of five LLMs across six shot‑count configurations (k in 0,1,2,3,5,8) on the AG News four‑class benchmark (n=200). Our models span proprietary and open‑source families: Gemini Flash Lite, GPT‑4o‑mini, Llama 3.1 8B, Llama 3.3 70B, and Llama 4 Scout 17B. We report macro‑averaged F1 with 95% bootstrap confidence intervals (B=10,000), permutation‑test p‑values, and Cohen's d effect sizes across all 30 configurations. Our findings reveal four qualitatively distinct behavioral regimes: (1) models already well‑calibrated at zero‑shot that show modest, statistically insignificant gains (Gemini, GPT‑4o‑mini); (2) models that undergo catastrophic zero‑shot failure but recover dramatically with a single example (Llama 3.1 8B, d=10.98, p<0.0001); (3) models optimal at zero‑shot that degrade monotonically with additional examples (Llama 4 Scout); and (4) models exhibiting a U‑shaped curve (Llama 3.3 70B: 0‑shot F1=0.907, 2‑shot F1=0.635, 5‑shot F1=0.785 with parser corrected). We additionally identify, diagnose, and correct a systematic parsing artifact that artificially deflated Llama 3.3 70B performance by up to 206%, constituting a methodological contribution to LLM evaluation practice. Our results demonstrate that the relationship between shot count and classification performance is not monotonic, not universal, and not predictable from model scale alone.
Authors:Daniel Commey
Abstract:
Restricting access to a dual‑use AI model is precautionary only if it delays harmful actors more than defenders. That condition varies across actors: a state agency or organized criminal group may obtain a substitute through theft, distillation, intermediated access, independent development, or a foreign release, while a small utility or open‑source maintainer may have no comparable route. We model a laboratory choosing among controlled access, a defender‑first window, safeguarded open weights, and minimally restricted open weights. Access inversion occurs when restriction gives an access advantage to adversaries that obtain effective substitutes faster than defenders. Asymmetric empowerment occurs when immediate release adds the most capability to populations least likely to possess a substitute. The policy ranking also depends on relative usefulness, opportunistic misuse, offense‑defense conversion, defensive spillovers, safeguard friction, and nonrecallable losses. A linear benchmark yields a unique adversary‑substitution threshold above which broad release overtakes control when the endpoint conditions hold. A defender‑first window has value when selected defenders deploy protection before adversaries catch up, and removable safeguards remain useful when they deter enough opportunistic misuse. A nonlinear implementation gives each release tier a nonempty policy region. Three nested 2,048‑point deterministic designs assess sensitivity to parameter bounds, and a separate grid examines actor‑specific deployment delays after release. Release, cyber‑evaluation, and incident‑response cases identify the quantities a release review should estimate: actor‑specific substitution times, marginal capability gains, deployment rates, defensive reach, newly enabled misuse, and nonrecallable losses.
Authors:Ayush Dwivedi, Ashvi Soni
Abstract:
Large language models (LLMs) have become a default choice for structured product attribute extraction in e‑commerce pipelines, with practitioners reporting widely varying performance across models, datasets, and prompting strategies. This paper presents a controlled empirical study comparing four prompting strategies ‑‑ zero‑shot, few‑shot, schema‑guided, and definition‑augmented ‑‑ across two production‑grade LLMs (GPT‑4o‑mini and Gemini 2.5 Flash) on the MAVE benchmark. We evaluate 6,400 attribute‑level predictions using both exact and fuzzy string matching, and conduct a rigorous noise audit of the ground truth labels. We formally decompose F1 variance across four experimental factors and find that evaluation methodology produces variance approximately 23 times larger than model choice and 5 times larger than prompting strategy choice. We further establish that the MAVE benchmark exhibits a 23.2% ground truth noise rate against modern LLM outputs. Paired permutation tests (B=10,000) confirm that the inter‑protocol F1 gap is highly significant (p<0.0001) and Cohen's kappa of 0.769 between protocols indicates substantial agreement. We conclude that for production attribute extraction pipelines, evaluation methodology and data quality dominate the impact of model selection and prompt engineering.
Authors:Joseph Fioresi, Fabian Caba Heilbron, Pankaj Nathani, Mubarak Shah, Kushal Kafle
Abstract:
Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross‑modal zero‑shot classification. These embeddings compress high‑level semantics into a single vector, which comes at the cost of primarily expressing a dominant semantics like main object while suppressing other important attributes such as camera angle or color tone. We propose a text‑conditioned transformation of visual embeddings that makes such attributes explicitly accessible. Given a natural language description of an attribute category (e.g., "color" or "art style"), a network generates an affine transformation that emphasizes the specified attribute. Conditioning on text enables it to learn many attributes simultaneously, accessing them at inference time through an intuitive interface. The network is trained to align transformed embeddings with the frozen latent space, enabling retrieval using existing large‑scale embeddings without any re‑encoding. When applied to a full set, the same mechanism transforms the latent space for attribute disentanglement tasks such as multi‑clustering. By operating directly in latent space, our method provides a unified and efficient framework for controlling embedding spaces, demonstrating state‑of‑the‑art performance across both attribute‑based retrieval and multi‑attribute organization tasks with near‑zero inference cost. Project page: https://joefioresi718.github.io/ControlEmbed_webpage/
Authors:Onur Onal, Chen Chen
Abstract:
Detecting pollinators in field video is challenging: targets are small, visually similar, and observed against cluttered vegetation under blur and occlusion. We present a systematic empirical study of small‑pollinator detection under a practical single‑GPU compute budget. Using the BuzzSpot challenge dataset, we compare YOLO and RF‑DETR models across input resolutions and evaluate sliced inference, class‑gated fusion, size‑routed ensembling, and post‑hoc temporal processing. RF‑DETR Large at 1344‑pixel resolution achieved our best hidden‑test result, reaching 0.405 mAP50:95 and outperforming the 1120‑pixel model (0.379) and the best single‑model YOLO26m baseline (0.366). The strongest gains came from adopting RF‑DETR and increasing its input resolution, indicating that detector choice and input resolution were more effective levers than added inference‑time complexity; the resolution gain was strongest for small objects and the rarer bumblebee and moth classes. Sliced‑inference fusion, size‑routed ensembling, and warm‑started 1536‑pixel continuation did not surpass this result, while post‑hoc temporal processing did not improve the leaked diagnostic evaluation. Error analysis identified bee‑hoverfly discrimination as the clearest remaining bottleneck: neighboring frames rarely supplied correctly classified hoverfly evidence for post‑hoc correction. These findings motivate learned feature‑level temporal aggregation before the final classification decision.
Authors:Ofek I. Cohen, Lior Shani, Aviv Rosenberg, Ankur Samanta, Tal Wagner, Yonathan Efroni
Abstract:
Many organizations aim to adapt language models for internal use, both to improve performance on domain‑specific tasks and to address privacy concerns around sensitive data. However, such adaptation remains non‑trivial: it often requires operationally challenging fine‑tuning of open‑source models or ad hoc prompt optimization. We study a minimal alternative based on a simple API‑level control: allowing users to bias the model's logits with a user‑defined vector. We develop a black‑box method for learning a single context‑independent logit‑bias vector, added at every decoding step, without modifying model weights or requiring gradients. Starting from a KL‑regularized reinforcement learning (RL) objective, we characterize when such a fixed logit‑bias vector can approximate the optimal prefix‑dependent correction and derive a closed‑form inverse‑propensity estimator from rollouts, rewards, and token probabilities. Empirically, this simple decoding‑time intervention improves over base models on mathematical and reasoning benchmarks while using far fewer trainable parameters than conventional fine‑tuning. Our results suggest that learned logit bias is a lightweight mechanism for adapting language models under minimal access requirements.
Authors:Alkis Sygkounas, Victor Aregbede, Amy Loutfi, Andreas Persson
Abstract:
Long‑horizon embodied tasks require policies that execute many dependent actions before task success can be observed. Representing policies as executable control pro‑ grams (code‑as‑policy) enables their decision logic to be inspected and revised after rollout evaluation. Revised programs can then be executed and compared by rollout performance, framing policy improvement as execution‑guided program search. Evo‑ lutionary methods driven by large language models (LLMs) provide a natural mecha‑ nism for this search by generating variants and selecting high‑performing candidates. However, existing approaches primarily select among independently generated vari‑ ants and lack a sequential local improvement phase. We introduce MEMENTO, a memory‑guided single‑elite memetic framework for code‑as‑policy evolution. ME‑ MENTO first evolves a rollout evaluator that maps policy rollouts to scalar fitness and structured feedback metrics. Fitness selects accepted candidates and the next elite, while feedback metrics condition policy proposals generated by memory‑guided hill‑climbing, macro‑mutation, and crossover. We evaluate MEMENTO on two long‑ horizon embodied domains: Robosuite Franka Tower‑of‑Hanoi manipulation and AI2‑ THOR household interaction. MEMENTO outperforms Eureka and REvolve, adapted as code‑as‑policy evolutionary baselines, in task success and generalization to held‑ out Robosuite object configurations and unseen AI2‑THOR scenes. Ablations show that zero‑shot generation and unevolved evaluators fail to solve either domain, and that removing policy‑search branches reduces performance. Finally, we deploy the best‑evolved Robosuite policy on a physical Franka robot, demonstrating the feasibil‑ ity of sim‑to‑real transfer of the evolved code‑as‑policy. Code, prompts, and videos are available at: https://github.com/sygkounas/MEMENTO.
Authors:Yuancheng Xu, Mingming He, Pablo Salamanca, Li Ma, Yash Kant, Emmett Steven, Paul Debevec, Ning Yu
Abstract:
In visual storytelling, human performances are central to creative intent and narrative meaning. However, preserving human identity and performance while enabling flexible visual edits remains challenging for generative video models. We formalize this challenge as identity‑preserving video restylization, which propagates scene, lighting, and style changes specified by an edited keyframe across a source video, while preserving facial likeness and performance, including expressions, eye gaze, and lip synchronization. A key obstacle is the absence of paired training data, as identity‑preserving restylized video pairs are rare in real‑world settings. To address this, we propose a decoupling of source‑grounded identity preservation and edit‑driven video synthesis. Our key insight is that facial appearance and expression should remain invariant, with illumination being the primary permissible variation. We therefore cast identity preservation as a video relighting problem, while modeling visual edit propagation as controlled video synthesis guided by the edited keyframe. Building on this formulation, we introduce ID‑V2V, a video‑to‑video generative framework integrating complementary control signals: relit facial regions and facial normal maps tightly constrain facial likeness and performance, while edited keyframes and depth sequences enable flexible and temporally coherent generation. This design enables constructing training pairs from a single video, eliminating the need for scarce paired data. Extensive experiments demonstrate that ID‑V2V significantly outperforms existing methods in preserving facial likeness and fine‑grained facial performance, supports both single‑ and multi‑subject scenarios, and delivers high visual quality, highlighting its potential as a human‑centric tool for real‑world content production. The code is available at: https://github.com/Eyeline‑Labs/ID‑V2V.
Authors:Xinglin Lian, Chengtai Cao, Ting Zhong, Fan Zhou
Abstract:
Network Traffic Anomaly Detection (NTAD) is a critical task in cybersecurity, yet timely and accurate anomaly detection remains challenging. Mamba has emerged as a particularly promising backbone for NTAD due to its linear‑time complexity for long‑sequence modeling. It further incorporates a dedicated multi‑view scanning mechanism to enhance detection precision through complementary contextual cues. However, we identify a previously overlooked structural deficiency in multi‑view Mamba scanning for NTAD: redundancy accumulation. Specifically, distinct scanning branches capture substantial view‑invariant information, which is repeatedly amplified during multi‑view fusion; conversely, view‑specific information is diluted or even suppressed, leading to representation homogenization and multi‑view degradation. To address this problem, we propose DisenMamba, a novel disentangled multi‑view Mamba framework. DisenMamba reformulates multi‑view scanning as a two‑stage disentangle‑then‑fuse process that explicitly separates view‑invariant and view‑specific components prior to fusion. This design prevents the invariant information accumulation while preserving complementary multi‑view cues, yielding more discriminative representations for subtle traffic anomalies. Extensive experiments demonstrate the effectiveness of DisenMamba, establishing a new disentangled multi‑view Mamba paradigm. Code is available at https://github.com/ikun0124/DisenMamba.
Authors:Ilia Sobakinskikh, Paul Alexander Bilokon
Abstract:
In this work, we explore how the inference time of a Transformer Neural Network can be efficiently optimized with applications to real‑time anomaly detection in financial time series. The financial time series are price series such as asset prices. Unfortunately, the data is often with errors or outliers that make the downstream data processing tasks useless, unstable or even harmful. Moreover, the amount of financial time‑series data has been significantly increasing. Hence, there is a need for better data‑cleaning methods in terms of accuracy and in terms of processing speed. Transformers as a neural network architecture have achieved superior performances in many tasks such as Natural Language Processing and Computer Vision. Time series modelling and especially anomaly detection tasks can benefit from the features of transformers architecture in multiple ways, including the capacity to capture long‑range dependencies and interactions. Increasingly powerful hardware, such as field‑programmable gate arrays (FPGAs), have seen increasing usage in recent years due to their reconfigurability and high performance. They can be efficiently utilized to speed up the computations of the Transformer architecture. We explore different Transformer architectures for time series modelling and how they can be efficiently implemented on an FPGA board (PYNQ‑Z2). In particular, we examine the application of Transformers to detect anomalies in time series and we show how they can be efficiently implemented on an FPGA board to minimize latency. The code is available at https://github.com/thxi/icl_thesis
Authors:Federico Del Pup, Elisa Tentori, Manfredo Atzori
Abstract:
Hand gesture recognition via surface electromyography (sEMG) is fundamental to prosthetic control. In this field, deep learning approaches have become the gold standard. However, current architectures struggle to scale; model performance typically decreases as the number of hand movements increases. Performance degradation is tied to the increased statistical complexity of decoding expanded gesture sets and compounded by the limitations of state‑of‑the‑art methods, which primarily rely on low‑latency unimodal convolutional architectures. Convolutions operate locally, limiting model's ability to capture long‑range sequential patterns. Unimodal setups cannot leverage complementary information from coordinated signals characterizing movement execution, such as inertial and eye‑tracking data. These limitations motivate architectures that integrate local and global features across multimodal physiological sequences. To bridge this gap, this study introduces EMG‑CrossFormer, an end‑to‑end hybrid convolutional‑transformer for seamless multimodal integration. EMG‑CrossFormer combines representations from an arbitrary number of unimodal encoders through cascaded cross‑attention fusion layers, and decodes the fused representations using learnable gesture queries. EMG‑CrossFormer was evaluated on four NinaPro datasets (DB2, DB3, DB7, and DB10) and benchmarked against six state‑of‑the‑art models using an increasing number of modalities. Using only sEMG, EMG‑CrossFormer achieved mean accuracies of 72.33%, 52.48%, 79.16%, and 73.49% on DB2, DB3, DB7, and DB10, respectively. Incorporating inertial signals improved performance to 90.66%, 80.40%, 92.79%, and 92.06%. These results show that joint local‑global feature modeling improves sEMG‑only decoding and that multimodal fusion substantially amplifies this benefit, underscoring the value of both design principles for complex hand gesture recognition.
Authors:Zixiang Tong, Jin Yang
Abstract:
pyALDIC is an open‑source Python implementation of augmented Lagrangian digital image correlation (AL‑DIC) for full‑field displacement and strain measurement. The software combines a graphical user interface with a scriptable Python API and supports adaptive quadtree meshing, mask‑aware subset splitting near cracks and holes, and selectable Local DIC and AL‑DIC solver modes. Numba acceleration enables efficient analysis, while automated tests, documentation, and reproducible examples support reliable use acrossWindows, macOS, and Linux. Verification cases include synthetic displacement fields, rigid‑body motion, Mode‑I cracking, adaptive refinement, and experimental uniaxial tension. pyALDIC is distributed through PyPI, GitHub, and Zenodo under a BSD‑3‑Clause license for reproducibility. pyALDIC is openly available at https://github.com/zachtong/pyALDIC.
Authors:Hongruixuan Chen, He Huang, Haifeng Wang, Jian Song, Junjue Wang, Weihao Xuan, Hamish Mitchell, Jiepan Li, Wei He, Liangpei Zhang, Zijie Wang, Chen Zhong, Jiazhen Zhao, Lei Hu, Ting Hu, Hongyan Zhang, Gregory Angelides, Miriam Cha, Clifford Broni-Bediako, Junshi Xia, Taylor Perron, Naoto Yokoya
Abstract:
Rapid post‑disaster response requires timely, building‑level information on whether structures remain intact, are damaged, or are destroyed. Post‑event optical imagery, however, may be unavailable because of cloud, smoke, or darkness. The Bright Challenge evaluated all‑weather building damage mapping from a submeter‑resolution pre‑event optical image and a post‑event SAR image. Participants were required to detect and delineate each building and assign exactly one of three mutually exclusive damage labels. The challenge extended the globally distributed \textscBright dataset with instance‑level annotations for about 291,000 buildings across 16 disaster events spanning seven disaster types. The final phase was evaluated exclusively on two 2025 events absent from training: a wildfire event in California and a hurricane in Jamaica. A total of 157 participants made 1,289 submissions, and 46 teams entered the final phase. The two winning solutions achieved test mAPs of 0.182 and 0.181, approximately 8.7 times the public baseline of 0.021, but remained far below the best in‑domain holdout score of 0.513. Across teams ranked in both phases, performance declined sharply and the rank order changed substantially. The two leading solutions independently favored modality‑specific encoding, staged or late optical‑‑SAR fusion, and an optical‑dominant separation of building localization from damage recognition. The winning method additionally used scene‑aware threshold adjustment and pseudo‑label adaptation. These results identify cross‑event generalization and stable severity discrimination as the principal remaining challenges. All data, annotations, baseline code, and winning solutions are publicly available at https://github.com/ChenHongruixuan/BRIGHT.
Authors:Zihan Song, Shuo Ye, Bo Zhao, Ruixin Zhang, Jiayu Zhang, Shouhong Ding, Zitong Yu
Abstract:
Despite advances in Video Large Language Models (VLLMs) that have displayed promising outcomes in video understanding, the redundancy in the long‑duration frames remains a hindrance to efficient reasoning. This paper introduces a training‑free \mathbfPersistence‑Aware \mathbfCompression and \mathbfAggregation (PCA) method designed to preserve high‑fidelity raw visual information before the encoding stage. PCA can be built on arbitrary VLLMs and consists of two modules: 1) A Dynamic Downsampling (DD) module that adaptively removes redundant frames by analyzing frame‑wise similarity. 2) A Persistence‑Aware Motion Enhancement (PAME) module that enriches each selected keyframe by aggregating the temporal context of its neighbors, ensuring that essential information is preserved even after aggressive frame reduction. Our approach substantially reduces the computation of long‑context modeling, while enhancing the performance of the baseline model. Extensive experiments demonstrate that PCA consistently outperforms existing state‑of‑the‑art approaches in both efficiency and accuracy, achieving a speedup of 1.8× to 2.5× compared to the baseline VLLM. The code is open‑sourced at https://github.com/Heisenberg10110/PCA.
Authors:Huafu Li, Guo Chen, Jia Xia, Lei Wang, Wei Du, Yun Yao, Weijun Peng, Liming Li
Abstract:
Visual information extraction (VIE) from visually rich documents remains challenging due to high layout variability and real‑world impairments. Existing methods typically rely on sequential OCR pipelines or end‑to‑end models requiring extensive labeled data and layout‑specific training, limiting their scalability.We propose a classification‑guided large vision‑language model (LVLM) framework for multi‑type VIE that achieves high accuracy with minimal supervision. The approach decouples document‑type classification from content extraction and employs in‑context learning (ICL)‑based dynamic prompt engineering to inject task‑specific knowledge, enabling robust zero‑shot inference across diverse layouts. From a theoretical perspective, the proposed method can be viewed as a form of conditional computation that reduces task uncertainty and improves information efficiency during prompt‑based inference. Evaluated on a real‑world bidding dataset with 16 certificate types, our zero‑shot method (based on Qwen2.5‑VL‑7B) outperforms a strong supervised baseline by 18.35 percentage points in F1‑score (86.43% vs. 68.08%) and 0.23 in normalized edit distance (0.90 vs. 0.67). Optional domain‑specific fine‑tuning further improves performance to 93.65% F1 and 0.93 NED, demonstrating superior robustness against seals, watermarks, and low contrast. The framework offers an efficient, scalable solution for complex document understanding in office automation. Code is available at https://github.com/FairmeHIT/Multi‑VIE, and fine‑tuned models at https://huggingface.co/fairme/Qwen2.5‑VL‑7B‑SFT.
Authors:Xuran Hu, Yujie Zhu, Tengxi Wang, Jilong Li, Wufan Zhao
Abstract:
Multiplicative Gamma noise is a signal‑dependent degradation in coherent imaging; synthetic aperture radar (SAR) despeckling is its most prominent real‑world instance. Existing diffusion denoisers parameterize their forward process by abstract signal‑to‑noise schedules rather than by the physical look number L, so different deployment scenarios typically require separately trained models, and transfer from synthetic Gamma training to real SAR remains challenging without clean ground truth. We introduce γ‑Bridge, a look‑parametric bridge whose schedule L(t) connects the noisy observation at L_obs to the clean limit through exact multiplicative Gamma marginals. Its closed‑form Gamma‑‑Lévy reverse posterior admits both stochastic and deterministic processes, while observation conditioning and a two‑step consistency loss stabilize multi‑step inference in the low‑SNR single‑look regime. Because bridge time directly represents L, one conditioned network can smart‑start from any admissible input look and stop at a target look number. These two orthogonal controls enable zero‑shot restoration over the full admissible grid after training only at L_obs = 1 on natural images with synthetic Gamma corruption. Combined with a homogeneous‑patch look estimator, γ‑Bridge processes data from six spaceborne and airborne SAR sensors without sensor‑specific fine‑tuning, achieving leading results on standard synthetic benchmarks while providing physically interpretable input and output controls absent from prior denoisers. Codes are released \hrefhttps://github.com/Teriri1999/GammaBridgehere.
Authors:Mohammad Arafat Hussain, Ellen Grant, Yangming Ou
Abstract:
We propose parameter‑efficient SSM‑based U‑Net architectures for 3D medical image segmentation. Convolutional U‑Nets afford O(n) local mixing per layer but lack explicit global context; transformers provide global reasoning at O(n^2) cost in sequence length n. State‑space models (SSMs), such as Mamba, offer O(n) global propagation per block. Yet, existing medical SSM segmenters rely on fixed scan patterns and large parameter budgets. Dynamic Adaptive Scan (DAS), which learns data‑dependent reordering before selective scan, has not been applied to medical imaging or extended to 3D volumes. We propose DAMamba‑UNet3D, a hybrid encoder‑decoder that integrates tri‑plane 3D‑DAS blocks at encoder stages E2‑E4 while retaining convolutions elsewhere (~5.3M parameters). On BraTS 2020 five‑fold cross‑validation, DAMamba‑UNet3D achieves mean Dice 0.815+/‑0.013 (full‑volume per‑case evaluation) at ~13x lower parameter cost than SegMamba (0.824+\‑0.014, ~70M). At comparable scale, DAMamba‑L (~70M), a wide DAS‑native variant with encoder‑only DAMamba and a convolutional bottleneck, reaches 0.829+\‑0.012, surpassing retrained SegMamba by 0.5pt. Component ablations show that encoder‑only DAS placement is critical as bottleneck and decoder SSM blocks lower Dice. Together, the results suggest that learned tri‑plane DAS in a hybrid U‑Net is competitive with, and under our large‑scale design may improve upon, SegMamba's fixed Tri‑orientated Mamba (ToM) scanning on BraTS 2020. Code: https://github.com/marafathussain/DAMamba‑UNet3D.
Authors:Shishen Gu, Jiequan Cui, Wenbo Hu, Zenglin Shi, Zhenzhen Hu, Richang Hong
Abstract:
In this paper, we show for the first time that visual token pruning enhances the robustness of Multimodal Large Language Models (MLLMs), mitigating vulnerabilities such as jailbreak attacks and hallucinations. Given that vision and language modalities cannot be perfectly aligned, the misaligned visual tokens might act as out‑of‑distribution (OOD) inputs, leading to unpredictable outputs and introducing potential vulnerabilities. Building on this insight, we aim to enhance model robustness against jailbreaks and hallucinations by reducing OOD visual tokens at robust‑pruning layers, while also reducing inference cost as a side benefit. Specifically, we measure the distance between each visual token and the language feature space. Then, visual tokens with large distances are identified as OOD tokens, which can be iteratively pruned. To demonstrate the effectiveness of our method, we evaluate it on seven diverse popular benchmarks. Notably, our method yields an average improvement of 13.29% in defending jailbreak attacks, consistently achieves competitive performance in mitigating hallucinations, and maintains strong results on general datasets like MME.
Authors:Jizhong Li
Abstract:
Single‑image reflection removal aims to recover a clean transmission layer from one image captured through glass. We study an explicit decomposition pipeline built on RDNet and introduce LowAux, a training‑only low‑pass reflection auxiliary objective. The original residual target remains the main reflection supervision, while symmetrically filtered prediction and target provide a stable low‑frequency constraint. We further incorporate scene‑balanced real pairs from RRW to broaden real‑scene coverage and improve cross‑dataset generalization. To avoid evaluation discrepancies caused by model‑specific resizing, padding, output quantization, and metric code, we build a unified public benchmark over CEILNet, Real20, Postcard, Objects, and Wild. Under the same evaluator, the proposed system obtains a five‑dataset macro average of 27.546 dB PSNR, 0.9220 SSIM, 0.9751 NCC, and 0.004760 LMSE, achieving the highest macro‑average PSNR, SSIM, and NCC and the lowest LMSE among the compared public checkpoints and internal variants. Per‑dataset and qualitative analyses show that the main benefit is a more balanced performance across diverse reflection distributions, while clear semantic reflections in Postcard remain challenging.
Authors:Hyewon Lee, Minkyung Song, Junghyun Oh, Seunghoon Han, Sungsu Lim
Abstract:
This paper presents the MPR‑CiteG framework, which achieved second place in the ScienceON AI Challenge by addressing two fundamental challenges in generative AI: inefficient retrieval and the absence of source verification. We propose a dual‑component system, termed MPR‑CiteG, in which the Multi‑Portfolio Retriever (MPR) efficiently retrieves diverse and relevant information, while the Citation‑Grounded Generation (CiteG) module ensures that every generated output remains factually consistent and explicitly attributed to its source. MPR‑CiteG represents a significant step toward building more trustworthy and accurate LLMs that are not only capable of generating information but also of grounding their responses in reliable evidence, thereby mitigating common issues like model hallucination. Extensive experiments on the challenge dataset validate the effectiveness and reliability of our approach. Our code is available at https://github.com/2noweyh/MPR‑citeG.
Authors:Anuraag Gadehothur Karnam, Tarunesh Sathish
Abstract:
Object‑centric learning aims to represent scenes as objects whose properties can be reused in new combinations. Existing evaluations usually score segmentation, single‑image factor prediction, or downstream accuracy, but these tests do not directly ask whether a per‑object representation behaves correctly under a controlled semantic edit. We introduce EditCLEVR, a paired‑scene intervention benchmark in which each example contains a before/after pair of CLEVR‑style renders with the same object indices and scene layout, and either exactly one known attribute change on one known object or a no‑edit re‑render for drift measurement. The protocol includes probe‑free diagnostics for representation‑change localization and stability, together with probe‑decoded semantic faithfulness metrics that test whether the predicted scene change matches the intended intervention across in‑distribution and compositional out‑of‑distribution (OOD) suites, allowing code‑space movement and decoded object‑attribute correctness to be evaluated separately. We introduce the semantic metric Scene‑Graph Intervention Accuracy (SGIA), which requires the full after‑scene prediction to be correct and the only predicted before‑to‑after semantic change to be the intended object‑factor edit. We also establish Delta‑SGIA as a companion diagnostic that checks the single‑site change pattern without requiring the full after‑scene graph to be correct. Baseline evaluations on ground‑truth‑mask backbones, learned‑slot models, SAM 2 + frozen‑ViT models, and one mask‑feature hybrid indicate that CoGenT‑OOD‑core degradation can persist under ground‑truth instance masks, that mask source accounts for part but not all of native performance, and that locality or stability alone can overstate semantic faithfulness. Code is available at https://github.com/torux‑bughunter/EditCLEVR.
Authors:Xiao Wang, Hao Si, Qiang Chen, Yu-Xiang Zhang, Beihe Zhang, Jianhua Yang, Qingquan Yang, Dengdi Sun, Wanli Lyu, Guosheng Xu, Jin Tang
Abstract:
Nuclear fusion has made significant progress in recent years and is expected to become one of the most important pathways to addressing global energy challenges. This paper focuses on observing plasma using visible‑light cameras, analyzing its spatio‑temporal motion cues, and predicting the two‑dimensional spatial distribution of light intensity, aiming to provide a foundational basis for future scientific experiments using deep neural networks. Specifically, we propose Delta‑InvFormer, a novel backbone network centered on a differential Transformer. The key insight is that by taking consecutive video frames as input, we can better capture the dynamics of the plasma. Moreover, spatial and temporal differential self‑attention effectively mitigates interference from noisy signals, ensuring high‑quality feature extraction. These features are then fused into a compact and informative representation, which is fed into a decoder network to predict the distribution. Based on real experimental data collected from the Experimental Advanced Superconducting Tokamak (EAST) large‑scale scientific facility, our results demonstrate that the proposed model not only significantly accelerates traditional methods for distribution prediction but also achieves competitive reconstruction accuracy. The source code of this paper will be released on https://github.com/Event‑AHU/OpenFusion
Authors:Jing Yu, Yibo Zhao, Jiaming Zhang, Xiang Li
Abstract:
Long‑term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query‑relevant content. The model is trained with supervised fine‑tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem‑4B achieves an LLM‑judge accuracy of 0.85, outperforming the strongest non‑oracle baseline while using only 213 answer‑context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target‑domain training and reduces mean latency relative to the prior query‑time baseline. Code is available at https://github.com/allacnobug/LazyMem.
Authors:Zedong Yu, Qianxing Li, Zhi Gao, Liuyu Xiang, Chenrui Shi, Yang Liu, Huiming Wu, Yujie Wei, Yuhao Fei, Yubo Fu, Zhaofeng He
Abstract:
Graphical user interface (GUI) agents are systems powered by large multimodal models (LMMs). They perceive screen state and execute user instructions through GUI actions such as clicking, typing, and scrolling on desktops and mobile devices. However, current agents scale poorly to long‑horizon tasks: actions incur costly LMM inferences, and performance degrades as context grows. Humans divide such workloads among collaborators who complete sub‑tasks in parallel. Yet parallel coordination among GUI agents has received little attention. To close this gap, we introduce ParaGUIBench, to our knowledge, the first benchmark dedicated to parallel execution and coordination of multiple GUI agents on separate desktop instances. It consists of three components: a multi‑device Docker infrastructure with a shared file system; a dataset of 233 tasks spanning six task categories; and an evaluation system with efficiency metrics, including step reduction ratio and token cost. We further introduce ParaGUI, a planner‑worker agent that decomposes GUI tasks and dispatches sub‑tasks to concurrent workers on separate desktop instances. On ParaGUIBench, ParaGUI reaches a 46.4% success rate, outperforming the strongest serial baseline (Claude Sonnet 4.6) by 12.9 points while using roughly half the steps and less than half the tokens. These results show that parallel execution can improve both success rate and efficiency on decomposable, long‑horizon GUI tasks, pointing to a direction worth further study.
Authors:Ruihan Gao, Joonghyuk Shin, Ava Pun, Jaesik Park, Wenzhen Yuan, Jun-Yan Zhu
Abstract:
Tactile graphics are a primary medium for blind and low‑vision (BLV) individuals to access non‑textual information. However, they are difficult to scale or personalize. While recent generative models have revolutionized visual content creation, they are optimized for screen‑based visual realism and fail to satisfy the haptic perceptual and physical fabrication constraints required for touch. We present the first integrated generative system that produces fabrication‑ready 2.5D tactile graphics directly from natural language prompts, jointly generating global base geometry, fine‑grained tactile surface textures, and standard‑compliant braille within a unified 3D‑printable representation. Our approach introduces fabrication‑aware techniques, including template‑guided relief generation, a fast diffusion‑based text‑to‑texture module for high‑resolution tileable normal maps, and strict base flattening to ensure tactile readability and printability, while supporting both automatic generation and interactive texture control. Extensive evaluations, together with in‑person user studies with BLV participants and blindfolded sighted participants using physically 3D‑printed outputs, show that participants consistently prefer our results over baselines. By extending generative graphics beyond screens to touchable reliefs, our work broadens access to generative AI for the BLV community and beyond.
Authors:Seonghak Lee, Junhee Cho, Jisoo Park, Min-Gyu Park, Jongmin Lee, Ju Hong Yoon, Junseok Kwon
Abstract:
We present URHead, a unified representation for high‑fidelity and animatable head avatars that fundamentally redefines mesh‑Gaussian integration. While mesh‑based methods offer precise geometric control but lack photorealistic detail, and Gaussian‑based approaches achieve photorealism but suffer from poor structural consistency, existing hybrid solutions fail to fully leverage their complementary strengths. Our key contribution is a UV‑space unification where both representations share a common UV parameterization. Through joint optimization with adaptive gaussian sampling, our method automatically learns to disentangle and allocate appropriate roles to each component. URHead maintains full parametric controllability while preserving subject‑specific details, and outperforms existing state‑of‑the‑art methods in reconstruction quality and animation consistency.
Authors:Qi Han Wong
Abstract:
We investigate whether large language models alter medical triage recommendations for identical symptoms when only the patient's socioeconomic status (SES) varies. Using three deployment‑tier models (Gemini 3.5 Flash, Claude Sonnet 4.6, GPT‑5.4‑mini), we hold a single neurological symptom profile fixed and vary the SES signal along two channels: explicit (insurance status, occupation, housing) and implicit (a US ZIP code, with no other socioeconomic information). All three models raise their emergency‑room (ER) referral rate for lower‑SES patients given the explicit signal (spreads of 13‑50 percentage points). The effect is in the protective direction: lower‑SES patients are sent to the ER more often, not less. The model's stated reasoning stays clinically near‑identical across conditions, so the shift is invisible to a reasoning‑trace audit. Critically, sensitivity to the implicit ZIP‑code signal is model‑dependent: Gemini infers SES from geography alone, shifting its ER rate by a pooled 11.4 points across six US ZIP‑code pairs (p = 1.4e‑7, same direction in 6/6 pairs), while Claude Sonnet 4.6 stays flat (‑0.1 points) and GPT‑5.4‑mini shows only a small difference that is not sign‑consistent (2.0 points, predicted direction in just 2 of 6 pairs), neither a reliable ZIP‑code effect, despite both responding to the explicit signal. This reveals an explicitness gradient in the signal: every model acts on socioeconomic status when it is stated outright, but only Gemini Flash acts on it when it must be inferred from a proxy as thin as five digits. We read this as a model‑specific difference rather than a size or cost effect. A single‑sentence system‑prompt instruction reduces but does not eliminate the effect (Gemini's gap between low‑ and high‑income ZIPs falls from 11.4 to 5.8 points). We release all code, prompts, and raw results.
Authors:Fan Lyu, Wenqi Zhang, Joost van de Weijer
Abstract:
Personalized multimodal large language models (MLLMs) aim to generate user‑specific responses, but existing methods mainly rely on profile‑level information and overlook diverse user preferences. We identify group preference collapse, where multi‑user personalized MLLMs become insensitive to individual preferences and drift toward dominant population‑level choices due to suppressed preference signals and unreliable preference use during generation. We propose PrefMoE, a preference‑centric framework that separates stable profile information from preference‑related representations. PrefMoE decomposes preferences into shared prototypes and personalized residuals, preserves individualized residuals with imbalance‑aware learning, counterfactual pseudo‑user augmentation, and residual decorrelation, and routes profile and preference factors through separate LoRA adaptation paths. Experiments across multiple MLLM backbones show that PrefMoE improves preference‑sensitive personalization while substantially reducing preference collapse. Project page: https://prefmoe.github.io/.
Authors:Samyak Jhaveri, Erel Kaplan, Tom Yotam, Le Chen, Tomer Bitan, Niranjan Hasabnis, Gal Oren
Abstract:
Modern compute‑intensive software must migrate across a changing ecosystem of accelerators, programming APIs, compiler stacks, and portability layers, including CUDA, OpenMP, OpenCL, and OpenMP target offload. Large language models and autonomous coding agents are increasingly proposed for such migration, but the field lacks reliable ways to measure whether they preserve the low‑level parallel semantics that make translations behaviorally valid, including thread indexing, synchronization, memory management, host‑device coordination, and API‑specific execution structure. We present ParBench, a kernel‑centric benchmark framework for evaluating LLM‑based parallel API translation under executable, reproducible conditions. ParBench fixes the surrounding build, run, and verification infrastructure through declarative benchmark specifications and asks models to translate only the computational kernels. It draws on multiple open‑source HPC suites and covers representative cross‑API translation directions among CUDA, OpenMP, OpenCL, and OpenMP target offload. To test whether success reflects robust translation rather than surface‑form memorization, ParBench includes AST‑driven, intended behavior‑preserving, baseline‑validated source augmentation. Evaluations on state‑of‑the‑art open and proprietary LLMs show persistent barriers to reliable parallel code translation, including direction asymmetry, multi‑file coordination, incomplete API adaptation, and uneven robustness to source‑level perturbations. Code is available at https://github.com/Scientific‑Computing‑Lab/ParBench.
Authors:Jinsong Shu, Chenyang Wu, Zhongle Xie, Baokun Wang, Lidan Shou
Abstract:
Key‑Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a major bottleneck due to the large number of visual tokens. Recent prefill‑stage KV selection methods estimate KV importance from prefilling statistics, implicitly assuming that prefilling‑time queries are representative of those encountered during decoding. We show that this assumption breaks down in multimodal inference, where decoding‑time queries exhibit substantially larger variance than prefilling‑stage representations, leading to unstable KV importance estimation under tight cache budgets. As a result, small ranking errors can disproportionately discard semantically critical visual tokens and degrade grounding and reasoning performance. We propose MM‑ShiftKV, a training‑free, decode‑aware and strictly prefill‑only KV selection method. MM‑ShiftKV approximates decoding‑time query behavior during prefilling by constructing variance‑expanded query proxies and estimates prompt KV importance based on their aggregated attention mass. Experiments on multimodal benchmarks demonstrate that MM‑ShiftKV consistently outperforms existing methods under strict KV‑cache budgets. Our code is available at https://github.com/zjuDBxAI/MM‑ShiftKV.
Authors:Zeyu Zhang, Ziqing Wang, Kaize Ding
Abstract:
MedLoCoMo is a Medical Long‑Context Memory benchmark for patient‑specific clinical reasoning over multi‑admission medical dialogue. Existing medical QA benchmarks largely test short context knowledge or single document grounding, leaving open whether LLMs can use, connect, and abstain over longitudinal patient histories. We build MedLoCoMo from deidentified MIMIC‑IV and MIMIC‑IV‑Note records by constructing admission‑level clinical packets, synthesizing grounded doctor‑patient conversations, and generating evidence linked QA items over single‑admission, cross‑admission, and adversarial unanswerable settings. The benchmark contains 100 patient timelines averaging 1,669.8 turns, 29.7 sessions, and 74,512.2 tokens per conversation. Across the evaluated baselines, cross‑admission reasoning is consistently harder than localized evidence use, even when models have long context windows or use external memory or retrieval methods. The code and MedLoCoMo benchmark release is available at https://github.com/leozzy13/MedLoCoMo for use and reproducibility.
Authors:Thieu Khang Nguyen, Thu Huong Dang, Truong-Son Hy
Abstract:
The Chinese Postman Problem with load‑dependent costs (CPP‑LC) arises in real‑world logistics and transportation systems where travel costs depend on vehicle load and energy consumption. In this work, we propose a hybrid optimization framework that integrates metaheuristic search with mathematical programming to efficiently solve CPP‑LC. The proposed method combines local search procedures with reduced mixed‑integer linear programming (MILP) models to balance exploration and intensification. In addition, we develop an Ant Colony Optimization (ACO) algorithm to enhance scalability on large instances. Extensive experiments on benchmark datasets demonstrate that the proposed framework consistently achieves high‑quality solutions and outperforms existing approaches in solution quality, while maintaining competitive computational efficiency. These results highlight the effectiveness of hybrid optimization strategies for complex, load‑dependent routing problems in practical applications. Our implementation is publicly available at https://github.com/HySonLab/MatCPP
Authors:Byungjun Kim, Taeksoo Kim, Hyunsoo Cha, Hanbyul Joo
Abstract:
Action‑conditioned video world models predict future observations from an initial observation and an action signal. In robotics, actions influence future observations through two distinct processes: they are first realized into robot motion by the robot body and controller, and the scene then responds through contact and object motion. Conditioning directly on action commands asks the world model to learn the realization process itself, while conditioning on logged future states leaks the interaction outcomes it is meant to predict. We propose robot‑factored world models, which move two robot‑specific factors outside the world model. First, action realization: each command is rolled through the robot's own controller and kinematics into a deployment‑available nominal trajectory, a middle signal that avoids both action‑realization learning and future‑state leakage. Second, robot rendering: this nominal trajectory is rendered through the robot URDF, factoring the robot's geometry, kinematics, and appearance out of the model and into explicit rendered robot geometry. To resolve depth ambiguity, we pair end‑effector depth with scene depth, giving geometric cues for contact and occlusion beyond image‑plane overlap. Together, camera‑aware static RGB/depth context and rendered robot geometry form a shared visual world‑model interface that stays consistent across viewpoints and robot embodiments, so the model sees the action only as visible robot geometry and learns how objects respond to it. Our experiments show that the rendered interface outperforms vector‑conditioned baselines and generalizes to unseen robot embodiments at inference. We further demonstrate that our model generates robot manipulation videos from human demonstrations by retargeting and rendering the hand motion as robot geometry.
Authors:Shing Ho J. Lin, Wenzhao Zheng, Dong Zhuo, Yuqi Wu, Jie Zhou, Jiwen Lu
Abstract:
Geometry Foundation Models (GFMs) have substantially advanced monocular 3D reconstruction, yet extending this capability to 4D dynamic understanding remains a fundamental challenge. Most existing motion perception methods (e.g., sparse tracking, dense point‑wise flow) treat motion as independent point‑wise displacements, ignoring the structured nature of physical motion. However, real‑world objects usually obey rigid‑body kinematics, and points thus usually move collectively, not in isolation. Motion itself possesses geometric structure: physical objects undergo a set of rigid‑body transformations governed by SE(3), rather than unstructured point‑wise displacements. Building on this insight, we propose SM4RT, a Structured Motion 4D Reconstruction Transformer for end‑to‑end 3D reconstruction and structured motion perception. SM4RT introduces Structure‑of‑Motion to represent scene dynamics, where scene motion is decomposed into a compact set of motion bases, each represented as a temporal sequence of 6D twists in SE(3). Dense scene motion is then recovered by sparse, time‑shared per‑pixel assignment weights over these bases, ensuring points on the same object share a common rigid‑body motion trajectory. SM4RT introduces a parallel motion geometry encoder and decoder that jointly infer 3D geometry, world‑coordinate motion, and scene kinematic structure in a single forward pass from monocular RGB video. SM4RT achieves strong motion reconstruction performance while preserving the geometric structure of scene motion.
Authors:Kaixiong Gong, Xin Cai, Bin Lin, Hao Wang, Yunlong Lin, Mingzhe Zheng, Bohao Li, Jian-Wei Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Xiangyu Yue
Abstract:
Unified multimodal models seek a shared visual token space that supports both multimodal understanding and image generation. Discrete methods unify the interface via a shared codebook, whereas continuous pipelines often rely on two disparate representations ‑‑ semantic features (e.g., ViT) for understanding and low‑level latents (e.g., VAE) for synthesis ‑‑ resulting in mismatched latent spaces. We propose Twins, a unified continuous token space formed by channel‑wise concatenating ViT and VAE features on the same token grid, so the sequence length is unchanged and attention cost does not increase. However, jointly modeling Twins in a Diffusion Transformer exposes a severe optimization imbalance: the model fits the ViT component well but struggles to match the VAE latent distribution. We trace this imbalance to three sources of heterogeneity: frequency bias, intrinsic dimensionality, and condition‑aligned vs condition‑independent uncertainty. To address it, we adapt a focal regression objective for flow matching that upweights large‑error VAE dimensions, better balancing optimization across the ViT and VAE components. On ImageNet, this yields up to 10.57 gFID gain over naive MSE loss without classifier‑free guidance. Twins also performs competitively on multimodal understanding benchmarks and improves reconstruction fidelity, narrowing the gap between understanding‑ and generation‑oriented representations.
Authors:Siyuan Huang, Pengyu Cheng, Haotian Liu, Tao Chen, Yihao Liu, Jingwei Ni, Shijie Zhou, Ziyi Yang, Gangwei Jiang, Mengyu Zhou, Yu Cheng, Xiaoxi Jiang, Guanjun Jiang
Abstract:
LLM training is shifting from manual design and annotation to interaction‑driven self‑evolution. However, existing self‑evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment‑bound methods obtain precise feedback but confine learning to narrow domains, while open‑ended self‑generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop. We identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open‑ended task variety. Leveraging this insight, we introduce Skill Self‑Play (Skill‑SP), a co‑evolutionary framework comprising a proposer, a solver, and a dynamic skill controller. Orchestrated via a reinforcement learning loop, these components co‑evolve in a continuous self‑play loop: the proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library. This interactive co‑evolution effectively bridges the gap between structured verification and open‑ended exploration. Empirical evaluations on tool‑use and reasoning benchmarks demonstrate that Skill‑SP, serving as a robust evolution engine, consistently pushes the performance ceiling of competent backbones while catalyzing striking turnarounds for initially misaligned models. Our code is available at https://github.com/Qwen‑Applications/skill‑self‑play.
Authors:Junye Ji
Abstract:
We formalize in Lean 4 the Kannan‑Bachem Smith normal form algorithm for nonsingular square integer matrices. The program returns S,U,U^‑1,V,V^‑1 and proves UAV=S, U^‑1SV^‑1=A, four inverse identities, the Smith divisibility conditions, and equality of S with a canonical reference matrix. Stabilization terminates because each recursive pass strictly decreases the binary size of the active pivot; the outer algorithm recurses on the lower‑right block. The computation also emits a flat trace of designated sign‑magnitude arithmetic leaves. Branch conditions, quotients, Bezout data, and matrix entries are taken from the recorded primitive runs. Composite phases form their traces by concatenating the charge lists returned by the executed children. Verified self‑delimiting codecs define the input and output sizes. Coefficient and work recurrences, closed by a kernel‑checked polynomial‑envelope calculus, give fixed polynomial bounds for both trace cost and the encoded length of the five output matrices. The theorem concerns these arithmetic primitives; structural operations and compiled Lean runtime are outside the model.
Authors:Jiyuan Tan, Vasilis Syrgkanis
Abstract:
Automating theoretical research is constrained not only by the generation of candidate results, but also by their reliable evaluation. A common approach is to close the research loop with a large language model (LLM) reviewer. However, such reviewers remain empirically unreliable: they may accept fabricated papers and detect them at rates close to chance (Bad Scientist, 2025). We present CausalForge, a framework for automated theoretical research in causal inference grounded in the Lean proof assistant. CausalForge combines Causalean, a foundational Lean library for causal inference containing 7,035 machine‑checked declarations developed with language‑model assistance under human design and review, with CausalSmith, a self‑improving agentic pipeline that selects research topics, proposes results, formalizes statements, constructs proofs, and presents the resulting artifacts for human inspection. Because a machine‑checked proof establishes only that a formal statement follows from its assumptions, not that the statement faithfully captures the intended scientific claim, the pipeline augments kernel verification with a statement audit that compares each formal theorem against the informal claim it is intended to express. We evaluate the system using artifacts produced by completed autonomous research runs. The source code, formal library, and run records are available at https://github.com/Jiyuan‑Tan/CausalForge.
Authors:Mihael Simonič, Xiaocong Li
Abstract:
The paper proposes a robot‑agnostic compliant‑control framework that extends the ROS control ecosystem with standardized joint and Cartesian command interfaces. It addresses a key limitation of existing control software: no reusable infrastructure for implementing compliant‑control algorithms across different manipulators while preserving a common interface to higher‑level applications. A plugin‑based architecture separates controller infrastructure from control‑law implementation. Generic wrappers use existing hardware abstractions to interface with different manipulators, while runtime‑loaded plugins implement only the control law. Command interfaces support joint‑ and Cartesian‑space references, stiffness and damping gains, nullspace targets, and feedforward terms, enabling variable impedance and diverse compliant‑control formulations. Robot kinematics and dynamics are computed from URDF models using Pinocchio. The architecture facilitates the development of compliant‑control strategies and enables the same implementation to be deployed across platforms unchanged. The complete framework, including reference controllers, high‑level task interfaces, and example configurations for various manipulators, is open‑sourced. The reference Cartesian impedance controller supports task‑dependent compliance by rotating translational and rotational stiffness and damping, allowing the principal compliance directions to be updated online according to local task geometry rather than remaining fixed in the robot base or TCP frame. This is particularly important in contact‑rich manipulation, where the desired directions of motion, constraints, and compliance directions may vary throughout task execution. Real‑robot experiments demonstrate task‑dependent compliance in contact‑rich manipulation, while simulations show portability across manipulators with distinct kinematic and dynamic characteristics.
Authors:Víctor Rincón Yepes
Abstract:
Do learned audio embeddings encode structure that nobody told them to encode? We probe four large pretrained audio models (AST, CLAP, BEATs‑bio and BirdNET) with a downstream task none of them saw during training: recovering phylogenetic distance from species vocalizations. If the geometry of the embedding space tracks the tree of life, the representation is picking up something deeper than the labels the model was optimized for. We run Mantel tests across two independent radiations. In 32 marine mammal species (1,754 recordings from the Watkins Marine Mammal Sound Database) the foundation models recover strong phylogenetic signal within the 26 cetaceans (CLAP r=0.82, BEATs‑bio r=0.82, AST r=0.74; all p<0.001), among the highest acoustic‑phylogenetic correlations reported for any taxon. Hand‑crafted MFCC features (105d) find nothing (r=0.040, p=0.338). The gap survives after PCA‑projecting every embedding down to 105 dimensions, so it is not an artefact of representation size. It also survives a partial Mantel test controlling for dominant frequency (partial Mantel r=0.404, keeping 97% of the variance explained), so it is not just pitch in disguise. We repeat the analysis on 20 bird species using the Jetz et al. (2012) phylogeny, and this time add BirdNET, a classifier trained end‑to‑end on around 6,000 bird species. The general‑purpose foundation models recover the signal again (AST r=0.55, CLAP r=0.52). The unexpected result is that neither BirdNET nor the bioacoustic BEATs‑bio beat them (r around 0.32 to 0.36). Matching the training domain to the target taxon does not, by itself, help. Pretrained audio embeddings carry evolutionary information across two independent radiations, and domain‑specific pretraining is not required for it to emerge.
Authors:Oriol Jiménez-Ayguadé, Antonio Agudo
Abstract:
Recent radiance field methods represent scenes with 2D primitives that offer surface alignment and efficient rasterization, from Gaussian disks to triangles, yet all rely on convex boundaries: curved and concave structures demand excessive primitives. We introduce Deformable Triangle Splatting, which augments each triangle with K control points per edge, each parameterized by a single learnable scalar displacement that shifts the boundary inward or outward, enabling non‑convex shape representation while preserving the three base vertices that define the 3D plane. To render these non‑convex primitives differentiably, we design a rasterization pipeline in the triangle's barycentric coordinate space, ensuring view‑consistent rendering. A winding number test determines whether each pixel lies inside the deformed primitive, and a window function controlled by two learnable parameters, sharpness and corner smoothness, together with a per‑primitive scalar opacity, produces the smooth opacity transition from interior to boundary. Validation is done in a variety of real‑world scenes, outperforming recent works based on non‑volumetric primitives in terms of visual quality and versatility while still achieving competitive rendering efficiency.
Authors:Yihao Xiao, Jialong Sun, Zitian Gao, Zeming Wei, Chutian Wang, Ran Tao, Jiaye Teng, Bryan Dai
Abstract:
For scale‑invariant deep networks, Hyperball‑style optimizers have shown strong performance in large‑scale training by fixing the norms of matrix‑valued parameters and normalizing updates. However, the source of their advantage remains unclear. Starting from the angular displacement between consecutive parameter states, we derive an angular effective learning rate that accounts for the parameter‑update angle, parameter norm, and update norm. We also show that the conventional norm‑based measure is a special case under parameter‑update orthogonality. We then decompose optimizer updates into radial and tangential components and analyze how radial updates affect one‑step angular displacement. Under the training configurations considered, numerical results show that the radial component has only a limited direct effect on the angular effective learning rate. It therefore cannot explain why MuonH converges more slowly than MuonWD early in training but overtakes it later. To further isolate the underlying mechanism, we devise a heuristic experiment that modifies only the learning‑rate schedule so that the dynamics of each optimizer reproduce those of the other. The results suggest that their main difference stems from the evolution of the effective step size rather than an intrinsically superior update direction induced by Hyperball. Our pretraining experiments further show that more aggressive learning‑rate decay can accelerate MuonH early in training but may impair its later performance. Thus, maintaining a constant angular velocity does not eliminate the learning‑rate‑scheduling problem; careful scheduling remains essential to realizing the potential of Hyperball‑style optimizers. Our code is publicly available at https://github.com/mangocrazz/hyperball‑may‑not‑be‑a‑free‑lunch.
Authors:Austin Rockman
Abstract:
Sample retrieval tools can help composers find harmonically compatible material, but querying from a fixed reference sample becomes less informative as arrangements evolve and the harmonic context shifts with each musical decision. We present Reflector, an interactive audio workstation that tracks harmonic combinations as they accumulate on the composer's timeline and adapts retrieval as the arrangement develops. The system is organized around a fixed interval‑class oracle: a hand‑designed table of weights that scores how pitch‑class content combines between sources. An encoder trained entirely on synthetic audio learns to approximate the oracle in a 128‑dimensional embedding space, where dot products stand in for compatibility scores at interactive speed. As the composer arranges material on a multi‑track timeline, a sweep‑line analysis discovers co‑sounding regions, computes oracle‑weighted centroids, and retrieves against the composite harmonic identity of the session as it evolves. Session centroids projected into a navigable 3‑D space reveal structural harmonic relations across the composer's body of work. This paper is a systems account: we give the design rationale for each architectural decision, characterize Reflector's behavior through intrinsic measurements on a working sample library, and describe the implementation. The characterization yields a central finding: the learned embedding preserves the kernel's pairwise judgments while covering the whole library, something the kernel cannot do when used directly as a retrieval rule, because the embedding's normalized geometry cannot express the degenerate solutions that direct scoring favors. The entire pipeline runs locally with no copyrighted training data. Reflector is free, and the training pipeline is open source.
Authors:Jie Deng, Heyang Wang, Changxin Wang, Junkai Shen, Hongyi Chen, Zhiping He, Hongxing Qi, Xudong Zhang, Jianyu Wang
Abstract:
Efficient processing is becoming increasingly important in infrared remote sensing, where satellite constellations produce large volumes of observations under constrained detector resolution, power, and downlink bandwidth. Multi‑frame super‑resolution (MFSR) offers a software‑based route to spatial enhancement, but its evaluation in infrared sensing remains fragmented across private datasets and ad‑hoc protocols. Existing benchmarks do not explicitly capture the thermal contrast, sensor noise, weak texture, and platform‑induced frame‑to‑frame variation that characterize infrared video. We introduce IR275K, a curated benchmark containing 594 infrared video sequences and 275,196 frames. It provides sequence‑level train/validation/test splits and a reproducible X4 evaluation protocol. As an initial architectural probe, we further evaluate CGMamba, a lightweight state‑space model with 10.90M parameters and 112.14G FLOPs. CGMamba combines 2D rotary position encoding (2D~RoPE) with center‑guided cross‑Mamba (CGCM) fusion for implicit multi‑frame reconstruction. It achieves 33.19dB PSNR, outperforming infrared single‑image super‑resolution references by 0.35‑‑0.52~dB at substantially lower computational cost. Ablation results show that removing 2D~RoPE from CGCM causes a 1.53dB drop and severe grid‑like artifacts. This indicates that explicit spatial anchoring is critical for stabilizing SSM‑based cross‑frame gating under infrared conditions. IR275K provides a reproducible foundation for accuracy‑‑efficiency evaluation of infrared MFSR methods, while the architectural analysis offers a concrete starting point for spatially aware SSM design under resource‑constrained infrared sensing. Dataset and evaluation resources are available at: https://github.com/InfraRecon7/IR275K.
Authors:Varun Gumma, Navonil Majumder, Soumitra Sinhahajari, Soujanya Poria
Abstract:
Large Language Models (LLMs) have significantly automated the process of scientific discovery over the past few years. However, existing systems share one core limitation: they generate and optimize ideas independently for either Quality or Diversity. This often leads to the generation of ideas in close proximity to one another or to a large set of trivial, unsound, or unclear concepts. In this work, we instead argue that research ideation should be treated as a conjunction of both objectives and framed as a Quality‑Diversity (QD) search. In line with this perspective, we introduce IDEAgent, a multi‑agent framework that manages the evolution of ideas through lineages. We jointly drive Quality using multi‑objective feedback for dedicated repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, their historical ancestors, and rejected proposals. To systematically evaluate this QD conjunction, we develop Yield, a joint metric that computes the largest set of mutually diverse ideas that satisfy a predetermined quality threshold. Finally, through evaluations across 32 topics spanning 8 domains of Computer Science, we show that IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving non‑zero Yield on 8x more topics. We further corroborate these findings through an analysis of quality improvements, showing that repair and refinement are crucial for building logical rigor and clarity while preserving non‑obviousness. To encourage future research on QD‑search‑based ideation, we open‑source IDEAgent at https://github.com/declare‑lab/IDEAgent.
Authors:Ziyao Huang, Shunkai Li, Juan Cao, Chenyu Li, Youliang Zhang, Zixiang Zhou, Cong Wang, Yuan Zhou, Qinglin Lu, Fan Tang
Abstract:
Recent advances in video diffusion models have spurred interest in human‑object interaction (HOI) video generation, which demands fine‑grained control over interaction logic beyond single‑subject animation. However, existing HOI methods rely heavily on explicit motion control, limiting scalability and generalization across diverse objects and interactions. In this study, we propose AgentHOI, a text‑driven HOI video generation following a thinking‑before‑generation framework that bridges the gap between high‑level textual intent and physical execution through multi‑agent reasoning over perception, interaction, and motion planning. Building upon the generated interaction plans, we further strengthen text‑driven motion understanding. We introduce an implicit text‑motion alignment strategy that distills text‑to‑motion priors into the video diffusion model, enabling robust HOI synthesis without explicit motion inputs at inference. Experiments show that AgentHOI significantly improves interaction naturalness, object appearance preservation, and adherence to complex textual instructions across challenging object‑centric scenarios such as wearing and riding. The code is available at https://github.com/bone‑11/agenthoi.
Authors:Sicheng Gao, Zhuyun Zhou, Yixuan Liu, Tong Shen, Zongwei Wu, Radu Timofte
Abstract:
Video super‑resolution (VSR) using large‑scale Diffusion Transformer (DiT) priors achieves exceptional perceptual quality but is often impractical due to the quadratic computational cost of processing dense spatio‑temporal token sequences. Existing efficiency‑oriented methods risk irreversible detail loss and temporal flickering, a vulnerability especially pronounced in one‑step diffusion models. To address this, we propose TRaM‑VSR, a Token Routing and Merging framework for adaptive token allocation, leveraging both context‑aware video priors and network‑level priors. First, token importance is estimated by fusing motion‑sensitive temporal cues with semantic text similarity, isolating dynamic objects and structural boundaries. Next, this importance is further calibrated and adjusted by an offline planner to guide routing across optimally grouped network blocks. Technically, within each routed group, structurally critical tokens are processed in a high‑fidelity local stream, while less informative tokens are aggregated into a compact global stream, both modulated by network depth and aligned with the multigranular nature of diffusion models. Extensive experiments show that TRaM‑VSR accelerates inference significantly while preserving state‑of‑the‑art reconstruction quality and robust temporal consistency. The code is available at https://github.com/Ree1s/TRaM‑VSR.
Authors:Yuheng Zong, Minghua Wang, Xin Zhao, Zhi-Hui Zhan, Antonio Plaza, Jon Atli Benediktsson
Abstract:
Remote sensing multimodal large language models (RS‑MLLMs) have improved general aerial‑image understanding. However, Earth observation applications require fine‑grained scenario specialization, constrained by scarce high‑quality scenario data and incomplete capability coverage. We formulate this adaptation as a capability‑gap‑driven post‑training problem and propose filling before advancing (FBA). Rather than relying on single‑stage supervised fine‑tuning (SFT) over target‑domain samples, FBA first fills prerequisite capability gaps before advancing toward scenario specialization. We instantiate FBA for coastal harbor understanding, a representative multi‑source scenario, by constructing CPRS (Coastal‑Port Remote Sensing), a three‑layer supervision dataset coupled with three ordered stages: (1) RS semantic anchoring for overhead‑view visual‑language alignment; (2) domain‑bridge convergence for shared RS priors across target and bridging scenarios under different modalities; and (3) evidence‑grounded scenario tuning for downstream performance. We construct HarborEval, an eight‑track diagnostic benchmark covering perception, spatial understanding, robustness, and generation. Under comparable training budgets, HarborEval increases from 57.95 with Direct‑SFT to 70.29 with FBA on LLaVA‑v1.5, and from 81.09 to 83.37 on Qwen3‑VL. FBA also outperforms Collapsed‑SFT and leads on harbor‑related VRSBench/RSVQA subsets and OpenEval. Stage‑wise and role‑replacement analyses validate progressive gap filling and stage‑specific roles. Public examples and release updates for CPRS, HarborEval, code, and trained weights are available at https://github.com/Z0ngL1ng/filling‑before‑advancing.
Authors:Wooyung Yun, Dongwook Kim, Soomok Lee
Abstract:
Accurate yet low‑latency depth is essential for radar‑camera perception in autonomous systems. Cameras provide rich appearance but lack metric scale, whereas automotive radar offers metric range but is sparse and noisy. Many pipelines are multi‑stage or depend on auxiliary annotations, increasing latency and limiting portability. We introduce JustDepth, a single‑stage radar‑camera depth estimator trained only with radar, camera, and single‑scan LiDAR. All radar returns are aggregated into a fixed‑width 1D representation, decoupling runtime from point count. A Height Fusion Block fuses modalities, a lightweight GNN propagates depth globally, and a training‑only confidence decoder stabilizes learning with zero test‑time cost. We mitigate stripe artifacts via simple augmentations and quantify them using the Vertical‑Horizontal Gradient Ratio (VHGR). On nuScenes, compared to recent state‑of‑the‑art methods, JustDepth maintains accuracy while reducing inference time by 39.7x and stripe artifacts by 66% as measured by VHGR.
Authors:Maria Peribañez, Javier Civera, Rudolph Triebel, Riccardo Giubilato
Abstract:
Visual localization becomes extremely challenging in planetary‑like terrains characterized by low texture, perceptual aliasing, harsh illumination, and sparse, weakly overlapping viewpoints induced by forward rover motion and unconstrained driving directions. Under these conditions, state‑of‑the‑art image‑to‑image and image‑to‑map matching pipelines suffer significant performance degradation. In this work, we propose a visual relocalization method that departs from classical correspondence‑based pipelines by directly estimating camera poses against a differentiable map representation built with 3D Gaussian Splatting (3DGS). Our key contribution is a geometry‑aware training strategy that combines photometric and geometric losses, where the geometric supervision is provided for the first time by combining multi‑view stereo (MVS) and LiDAR depths. We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry, leading to improved photometric and geometric consistency and more robust, accurate single‑image 6‑DoF pose estimation. Extensive experiments on data acquired in planetary‑analog environments validate the effectiveness of our approach, showing substantial gains in relocalization accuracy under challenging conditions. Code is available at https://github.com/DLR‑RM/multimodal‑gsplat‑relocalization.
Authors:Dominik Bernard Lau, Hubert Malinowski, Jerzy Szyjut, Adam Brzeski, Tomasz Dziubich, Radosław Targoński, Tomasz Figatowski, Natalia Zielińska
Abstract:
Accurate pixel‑level classification of coronary angiograms is critical for cardiovascular disease assessment, yet the field lacks standardized evaluation protocols. In this work we demonstrate a new benchmark for the assessment of deep learning models which densely classify pixels of coronary angiograms to one of SYNTAX classes (or background). The evaluation covers 24 distinct architectures starting with classic convnets to recent state‑space‑based vision algorithms. We release CARDIAG ‑ a multi‑center, multi‑label dataset which we carefully split to reliably compute metrics, accounting for diameter error, overlap, centerline quality and calibration. The data contains SYNTAX labels, binary, uncertainty and segmentation masks as well as intermediate frames together with the selected non‑sensitive DICOM metadata. From the multitude of algorithms, we nominate ConvNeXt V2 encoder with DeepLab V3 Plus decoder as the best performing, achieving macro F_1=0.456, which we then ensemble with Mamba U‑Net and Feature Pyramid Network, for an increased F_1=0.479. We demonstrate all the architectures to be well calibrated and determine the generalization of the top 5 methods, together with the data efficiency of these architectures. We highlight the importance of both high‑resolution and low‑resolution features in encoding. We also demonstrate the model correctness in the context of patient demographic, vessel sides and projection angle configurations. Overall the released benchmark allows for future studies to robustly and rigorously assess the proposals, not only for SYNTAX segmentation, but lesion detection and many more.
Authors:Zhengyu Qi
Abstract:
Empathetic Response Generation (ERG) requires models to recognize users' emotions and generate empathetic responses. Commonsense knowledge has been shown to support such reasoning, yet existing approaches typically reuse fixed commonsense representations across understanding and generation, limiting their ability to coordinate such knowledge across different stages. We propose DCC, a Dynamic Commonsense Coordination Framework with three complementary modules: residual‑based commonsense interaction (SCE‑AttnRes) to integrate contextual and situational commonsense representations, Association‑Guided Commonsense Filtering (AGCF) to down‑weight low‑relevance commonsense relations, and Iterative Commonsense‑Aware Decoding (ICAD) to dynamically retrieve commonsense memories during generation. Experiments on the Empathetic‑Dialogues benchmark show that DCC improves emotion classification accuracy and response diversity over the CEM baseline while maintaining comparable perplexity. An LLM‑based blind evaluation further demonstrates that DCC generates responses with better relevance, coherence, and informativeness. The code and implementation details will be publicly available at https://github.com/Hanabi‑Q/DCC‑ERG.
Authors:Yuya Kobayashi, Masato Ishii, Yuhta Takida, Takashi Shibuya, Yuki Mitsufuji
Abstract:
Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency‑dependent discrepancies between training and inference, which can be interpreted as frequency‑dependent SNR error. Crucially, the direction of this mismatch varies across models and timesteps, indicating that fixed correction rules do not generalize. We propose Spectral Alignment (SPA), a lightweight, guidance‑based method that calibrates the power spectrum of intermediate predictions to a pre‑computed prior. Our approach consists of two stages: (1) offline fitting of a parametric spectrum model from training data, and (2) inference‑time guidance via efficient FFT‑based gradient computation. SPA introduces minimal computational overhead (3‑4%) and is complementary to Classifier‑Free Guidance (CFG). We demonstrate consistent improvements across diverse architectures, from pixel‑space models (DDPM, ADM) to latent diffusion models (SD2.0, SDXL) and flow‑matching models (SD3.5, FLUX). Our implementation is available at https://github.com/SonyResearch/SPA.
Authors:Boris Tokic, Constantin Selzer, Fabian B. Flohr
Abstract:
As autonomous driving systems move toward real‑world deployment, interpretable, behavior‑level decision‑making is essential for safety, trust, and regulation. We introduce CommandLM, a multimodal large language model that generates concise, human‑readable behavior descriptions for ego vehicles from fused multi‑sensor data. Our model processes temporally fused bird's‑eye view representations from LiDAR and multi‑camera inputs via a Q‑Former adapter connected to a quantized, LoRA‑fine‑tuned large language model. Trained on our CommandLM‑nuScenes dataset, CommandLM produces intent‑aware, interpretable captions suitable for planner supervision and safety auditing. Experiments demonstrate strong linguistic and behavioral alignment, achieving CIDEr 0.67, and BERT‑F1 0.88, substantially outperforming the BLIP‑2 baseline (CIDEr 0.52, BERT‑F1 0.86). In human evaluation, 58% of the generated descriptions were rated accurate, efficient and rule‑compliant, confirming their real‑world plausibility. While the remaining descriptions may not always select the most efficient, goal‑oriented behavior, CommandLM's interpretable outputs enable downstream validation systems to identify and correct such cases, making it an effective tool for transparent behavior auditing. These results show that integrating multimodal fusion with language reasoning yields efficient and transparent behavior‑level understanding for autonomous driving. We release our code and dataset at: https://github.com/b‑tok/CommandLM
Authors:Yuyuan Han, Jingwei Li, Long Qiu, Chong Wang, Wenxuan Hao, Jiangyu Han, Xinyu Yao, Yuchen He, Hui Chen, Jianbin Liu, Huaibin Zheng
Abstract:
Single‑pixel sensing encodes a scene as a short sequence of coded measurements, and image‑free methods infer the task directly from that sequence. Removing reconstruction does not remove the difficulty: it relocates it to the lift, the map from 1D measurements to a 2D representation, which prior work treats as a trivial reshape. We recast the lift as the central design axis of image‑free sensing and order methods by how strongly it adapts to its input: a fixed‑physics inverse (reconstruct‑then‑segment), a learned static projection, or a content‑adaptive retrieval; position on this lift spectrum predicts behavior as acquisition degrades. The spatiotemporal soft‑fusion (STSF) network pairs a probe‑selected recurrent encoder with a cross‑attention lift chosen by a parameter‑matched ablation, ahead of its U‑Net++ decoder, and trains under task‑prioritized loss scheduling (TPLS), a scheduled reconstruction prior. In simulation, STSF+TPLS surpasses the prior image‑free baseline on three datasets at 3.13% sampling (+3.2 to +9.9 pp foreground mIoU) and plateaus down to 0.39%. The strongest clean‑trained reconstruct‑then‑segment baseline wins the noiseless limit, but under calibrated measurement noise image‑free inference overtakes it, for a measured reason: the reconstruction pipeline amplifies the identical measurement noise before its segmenter reads it. Each region fails in its own signature: collapse, imprinting, or coarsening. STSF+TPLS transfers without fine‑tuning to a real single‑pixel bench as a proof of concept, at about 14 ms per mask. Charting the lift turns a scattered design space into a map of which lift to deploy at each operating point. Code and pretrained weights: https://github.com/Hanyuyuan6/STSF‑TPLS
Authors:Hao Yang, Jin Wang, Xuejie Zhang
Abstract:
MEMEs are widely used on the internet and often carry strong elements of sarcasm or irony. Understanding their hidden meanings typically requires a joint interpretation of text and vision. Existing methods focus on the dual‑stream vision‑language model to extract the visual and text simultaneously, which lacks background information and prior knowledge about the comprehensive explanation of MEME. One feasible option is to adopt chain‑of‑thought (CoT). However, the simple CoT approach lacks multi‑perspective thinking, which may compromise the reliability of the resulting answers. Moreover, it often relies on shallow feature fusion, lacking the fusion of local details and fine‑grained visual‑prompt text alignment. This limitation prevents a deeper understanding of the intricate connections between the visual and the text. Herein, an enhanced vision‑language multi‑CoT (EVL‑MCoT) approach is proposed to address these limitations. By promoting multi‑CoT, EVL‑MCoT enhances consistency and reduces bias in the decision‑making process. Additionally, we design a prototype‑guided and context‑guided decoding framework, which incorporates visual prototypes to guide the fusion process and enables the model to align textual and visual information more precisely. We achieve promising results on the HatefulMemes and MultiOff datasets. The source code has been publicly released and is available at https://github.com/BGWH123/EVL‑MCoT.
Authors:Hao Yang, Jin Wang, Xuejie Zhang
Abstract:
Multimodal chain‑of‑thought (CoT) reasoning integrates visual and textual cues through step‑by‑step inference. In small models with limited token budgets, modality‑interaction fusion often suppresses tiny cross‑modal differences. In particular, multimodal CoT often struggles when different images pair with identical text or different texts pair with an identical image, making such inputs nearly indistinguishable after fusion. This study proposes Visual Saliency Steering Distillation (VSSD). VSSD leverages the attention maps of multimodal large language models to generate perturbed images that capture task‑sensitive feature directions, and then applies singular value decomposition to extract dominant steering vectors to guide inter‑layer distillation. Experiments on ScienceQA and M^3CoT demonstrate that VSSD improves rationale generation and answer inference. The code is available at https://github.com/BGWH123/VSSD.
Authors:Jinhyeok Kim, Yejoon Lee, Jaeyoung Do
Abstract:
The increasing deployment of large language models (LLMs) has magnified the computational and memory bottlenecks of autoregressive decoding, where low compute intensity and bandwidth‑bound kernels dominate inference cost. Weight pruning offers a promising remedy, but existing methods remain confined to either static pruning (SP), which permanently removes redundant weights but lacks adaptivity, or dynamic pruning (DP), which adapts to input sparsity but introduces runtime irregularity. This paper presents SPDP, a unified sparse‑inference framework that integrates unstructured SP with input‑adaptive DP for efficient LLM inference on GPUs. SPDP co‑designs a new Tiled‑Column‑wise Bitmap Compressed (Tiled‑CBC) format and two complementary GPU kernels: (1) a CUDA‑core spMspV kernel featuring Hybrid Activation‑aware Dynamic Shared‑Memory Bitmap Decoding (HAD‑SMBD) for fine‑grained, runtime activation skipping, and (2) a Tensor‑Core SpMM kernel optimized for prefill computation. This joint format‑kernel design harmonizes static and dynamic sparsity, maintaining bandwidth‑efficient memory access and high compute intensity under both phases of LLM inference. Comprehensive evaluations on inference‑optimized GPUs demonstrate that SPDP achieves 1.24x‑1.37x average speedup (up to 2.51x) over state‑of‑the‑ art sparse frameworks such as SpInfer, while matching. perplexity with up to 25% higher sparsity. SPDP advances the inference efficiency‑quality Pareto frontier, showing that unified static‑dynamic pruning can deliver substantial throughput and performance‑per‑watt improvements in large‑scale LLM serving
Authors:Yuqi Li, Xi Xiao, Yunbei Zhang, Lin Zhao, Yu Li, Aiden Zhao, Tianyang Wang, Hao Xu, Yingli Tian
Abstract:
Vision foundation models are increasingly reused as frozen backbones for downstream visual recognition, making parameter‑efficient adaptation a central problem. Prompt‑based adaptation, including Visual Prompt Tuning (VPT), provides a lightweight way to specialize these models, but its layer‑wise behavior remains poorly understood: performance is sensitive to prompt depth, placement, and task distribution, and gains on standard in‑domain benchmarks do not always translate into robust generalization. We argue that this limitation is not solely an optimization issue, but a layer‑wise information allocation issue: existing prompt‑based methods lack principled control over what prompt‑conditioned representations should preserve, suppress, and propagate across depth. Inspired by the Information Bottleneck principle, we introduce Prompted Information Bottlenecks (PIB), a framework that regularizes layer‑wise compression‑sufficiency trade‑offs and promotes a more coherent cross‑layer information path. The key idea is that effective adaptation should be minimal yet sufficient, retaining task‑relevant local evidence in earlier layers while progressively discarding nuisance factors and redundant details in deeper layers. Extensive experiments show that PIB achieves strong performance across 34 datasets, reaching 92.1% on FGVC, 93.01% on HTA, and 77.33% on VTAB‑1k, while tuning only 0.35% parameters on average across the main settings. Beyond benchmark accuracy, PIB helps explain the non‑monotonic behavior of prompt capacity scaling, reduces shortcut reliance, and improves robustness under distribution shift and fine‑grained recognition settings. These results position PIB as both a practical method and an information‑allocation perspective for adapting frozen vision foundation models. Our code is available at https://github.com/itsnotacie/MM‑26‑PIB
Authors:Quentin Spencer
Abstract:
Benchmarks for LLM‑agent memory typically generate conversations first and extract answer keys afterwards ‑‑ with documented label‑error and contamination problems ‑‑ and they overwhelmingly measure short interaction histories. We invert the pipeline: a seeded life‑script sampler emits facts with validity intervals, volatility classes, and source channels before any text exists; an LLM renderer writes chat and email from per‑event fact manifests; a fidelity verifier confirms every planted fact; and questions are instantiated mechanically from the script, so gold answers are script‑valid by construction and separately validated for answerability. The synthetic, fictionalized corpus (~380 questions, 15 types) embeds features absent from the benchmarks we survey: per‑fact validity intervals, sent/received trust distinctions, injection probes in a benign harness, and as‑of‑date question sets. Benchmarking five memory architectures against a no‑memory control (fixed answerer, versioned LLM judge, three replicates, two horizons), we find backend rankings invert with history length: the budgeted curated‑map memory that leads at three weeks loses recall of evicted content by nine weeks (96% to 72%) while a provenance‑typed graph rises to 90%; the inversion is positive for all six users under complete cross‑family re‑judging (exact p=0.031). A full‑rendered‑history baseline ties or exceeds the best memory system at the short horizon but shows no judge‑independent advantage at nine weeks, at about twice the read cost. Write‑stage quality strongly correlates with downstream quality (weakly‑written facts fail 24% vs 2%), and injection resistance tracked whether provenance boundaries survive representation. A layered architecture performs best among the memory systems in both regimes (96.8% short‑horizon) and is released as Veracium, an open‑source library, with the corpus generator and harness.
Authors:Yining Yang, Ruogu Chen, Jie Han
Abstract:
Real‑time score following from sheet images remains chal‑ lenging because the model must process streaming au‑ dio while resolving highly repetitive visual patterns un‑ der strict latency constraints. Recent image‑based meth‑ ods have attempted to use multi‑resolution prediction by simultaneously predicting the positions of the active sys‑ tem, bar, and note. However, their predictions across these different levels of notation are independent, which makes the predictions unstable and introduces unnecessary ex‑ tra search space for bar‑ and note‑level predictions. Most existing methods also lack mechanisms to recover from score discontinuities, such as repeats, da capo (D.C.), or coda jumps. This paper proposes CODA, to the best of our knowledge, the first real‑time score following system that addresses both gaps. CODA explicitly exploits the cascaded structure of music scores: it first selects the ac‑ tive system, then the active bar within it, and finally the active note within the selected bar. This enforces pre‑ diction consistency across resolutions. A silence‑driven break mode enables recovery from arbitrary score discon‑ tinuities without requiring knowledge of the repeat struc‑ ture. Evaluated on the Multimodal Sheet Music Dataset (MSMD) piano benchmarks, CODA achieves state‑of‑the‑ art tracking accuracy and discontinuity‑recovery perfor‑ mance under real‑time throughput. Code is available at https://github.com/ValleyC/CODA.
Authors:Liangqin Ren, Zeyan Liu, Fengjun Li, Kaitai Liang, Zhu Li, Bo Luo
Abstract:
In the past decade, we have witnessed an exponential growth of deep learning models, platforms, and applications. While existing DL applications and Machine Learning as a service (MLaaS) frameworks assume fully trusted models, the need for privacy‑preserving DNN evaluation arises. In a secure multi‑party computation scenario, both the model and the data are considered proprietary, i.e., the model owner does not want to reveal the highly valuable DL model to the user, while the user does not wish to disclose their private data samples either. Conventional privacy‑preserving deep learning solutions ask the users to send encrypted samples to the model owners, who must handle the heavy lifting of ciphertext‑domain computation with homomorphic encryption. In this paper, we present a novel solution, namely, PrivDNN, which (1) offloads the computation to the user side by sharing an encrypted deep learning model with them, (2) significantly improves the efficiency of DNN evaluation using partial DNN encryption, (3) ensures model accuracy and model privacy using a core neuron selection and encryption scheme. Experimental results show that PrivDNN reduces privacy‑preserving DNN inference time and memory requirement by up to 97% while maintaining model performance and privacy. Codes can be found at https://github.com/LiangqinRen/PrivDNN
Authors:Guoming Li, Jian Yang, Xukun Wang, Zixiao Wang, Shangsong Liang, Yifan Chen
Abstract:
Coarsening‑based training for graph neural networks (GNNs), i.e.\ training on coarsened graphs rather than the original large ones, has become a promising direction for scaling GNNs to massive graphs. However, prior work has been evaluated almost exclusively on homophilic graphs, leaving the more challenging heterophilic settings underexplored. We show, both empirically and theoretically, that existing coarsening‑based training methods suffer significant performance degradation on heterophilic graphs due to inevitable loss of graph information during coarsening. To address this, we propose \bf Adaptive \bf Complementary \bf Enhancement, a plug‑and‑play, model‑agnostic strategy that reintegrates the information discarded in coarsening: ACE learns a projector for re‑constructing original node features and applies anisotropic structural regularization to embed local heterophily. We further adopt homoscedastic uncertainty weighting to adaptively balance the combined training objective of primary coarsened‑graph training loss and full‑graph auxiliary loss with augmented node features re‑constructed by the heterophily‑aware projector. Extensive experiments show that ACE drives consistent gains on heterophilic benchmarks while preserving competitive results on homophilic graphs with minimal computational overhead. Code is available at the GitHub repository: https://github.com/vasile‑paskardlgm/ACE.
Authors:Mohammadreza Narimani, Vikram Anand, Parastoo Farajpoor
Abstract:
Agricultural field maps are often proprietary, incomplete, or outdated, yet they provide the spatial framework for crop monitoring, production accounting, and land‑conversion analysis. This study presents a reproducible workflow for mapping farmland extent and visible boundaries from 1 m NAIP RGB imagery. Thirty‑seven scenes spanning open cropland, peri‑urban interfaces, semi‑arid irrigation geometries, and fragmented mosaics were annotated in CVAT and converted to binary masks. Non‑overlapping 256 x 256 patches yielded 5,698 samples, split by source scene into 3,850 training, 770 validation, and 1,078 test patches. A residual U‑Net (ResUNet) trained with a Dice‑dominant loss, L = 2.5(1 ‑ Dice) + BCE, achieved test accuracy 0.8808, IoU 0.8605, Dice 0.9234, precision 0.8766, and recall 0.9794. A frozen SAM 3 branch prompted with "agricultural farmland field" was fused with ResUNet by logical OR. On selected difficult patches, Dice improved from 0.858 to 0.955 (orchard rows) and from 0.804 to 0.903 (fragmented parcels). Sliding‑window stitching produced coherent regional masks (example tile Dice 0.898 and 0.919). The product is a semantic farmland‑extent layer, not a cadastral parcel map, and supports agricultural monitoring where current field layers are unavailable.
Authors:Hao Zhang, Yiwen Zhao, Yixuan Zhang, Yiwen Shao, Steve Yves
Abstract:
We present an agentic soundscape construction framework for controllable compositional audio generation that makes explicit the scene planning, source selection, temporal layout, and rendering steps typically handled implicitly by single‑shot text‑to‑audio models. An LLM‑based agent converts user intent into an executable scene plan, acquires assets through retrieval and on‑demand generation, renders controllable multi‑event mixtures, and exports aligned scene metadata. The framework also supports human‑in‑the‑loop interaction through user‑guided tool selection and editable scene plans. Together, these components provide an inspectable and reusable approach to controllable soundscape synthesis and scalable audio‑language data construction. Listener studies and objective metrics demonstrate competitive generation performance against text‑to‑audio baselines, while models trained with agent‑generated data consistently outperform real‑only baselines in downstream audio reasoning. Code, demos, and listening‑test results are available at https://haozhang6720.github.io/SoundscapeAgentDemoPage/.
Authors:Bertil Braun
Abstract:
We present a traffic‑signal control interface in which a shared graph neural network assigns scores to individual traffic movements. Each junction converts these scores into its own variable‑sized set of legal signal phases using a deterministic incidence matrix. Directed corridor nodes provide traffic context, while movement nodes represent controlled input‑to‑output paths through junctions. Typed mean aggregation produces one scalar per movement; phase definitions and signal timing remain outside the learned network. This makes graph size and junction‑specific action count independent of the learned parameter shapes. PPO experiments evaluate the interface on unseen synthetic grid geometries, altered signal coverage, and five heterogeneous city graphs. The policies retained performance across unseen geometries within the synthetic grid family, while changes in signal coverage exposed sensitivity to a signal‑coverage distribution shift. A single trained city‑policy instance executed across all five city graphs, with heterogeneous outcomes. These results provide feasibility evidence rather than a general estimate of transfer to arbitrary road networks.
Authors:Erencem Ozbey, Fethiye Irmak Dogan, Jin Huang, Hatice Gunes
Abstract:
Social appropriateness in human‑robot interaction (HRI) is not universal: different people can judge the same robot action differently in the same situation. To capture this inter‑subject variability, we reformulate socially appropriate action generation as a preference modelling problem inspired by recommender systems, treating annotators as users, contexts/scenes as items, and appropriateness scores over a set of candidate robot actions as targets. We propose StARS, a novel model‑agnostic framework that integrates collaborative filtering with learnable scene representations to generate user‑specific appropriateness scores over candidate robot actions. StARS is model‑agnostic: it can be integrated with various scene encoders and backbones, enabling personalisation without redesigning the underlying model. We evaluate StARS on two socially aware robotics datasets, MannersDB+ and SocNav1, and analyse robustness under sparse preference feedback. Across datasets and backbones, StARS consistently improves performance and agreement with annotators, supporting personalised action selection aligned with user norms. Our code is publicly available at https://github.com/Cambridge‑AFAR/StARS.git.
Authors:Jingguo Qu, Xinyang Han, Xiang Wang, Yuqi Yang, Tonghuan Xiao, Sheng Ning, Jing Qin, Ann Dorothy King, Winnie Chiu-Wing Chu, Jing Cai, Michael Ying
Abstract:
Medical ultrasound (US) image segmentation faces significant challenges due to speckle noise, low‑contrast boundaries, acoustic shadowing, and acquisition variation across operators and clinical centers. Although encoder‑decoder and transformer‑based networks have achieved strong performance, many methods recover boundary details through dense decoders or larger backbones, which may still produce over‑smoothed contours or unstable predictions under external distribution shifts. In this article, we propose Risk‑routed Implicit Boundary Refinement (RIBR), a compact segmentation framework that uses implicit neural representation as a risk‑routed residual correction rather than an unconstrained full‑mask predictor. RIBR combines boundary‑refinement implicit residuals, risk‑routed residual control, and geometry‑ and speckle‑aware boundary regularization to refine uncertain contours while suppressing non‑boundary oscillations. Evaluation on nine US datasets covering lymph nodes, breast lesions, thyroid nodules, and prostate shows that RIBR achieves the best overall macro‑average and consistently reduces boundary error across grouped and organ‑specific comparisons under a compact parameter budget. These findings suggest that controlled implicit residual learning is a practical strategy for resource‑constrained and boundary‑sensitive US segmentation. Source code is available at https://github.com/jinggqu/ribr.
Authors:Jianshu Zhang, Keliang Wu, Haoran Lu, Anbang Liu, Ce Zhang, Weijie Yin, Chengxuan Qian, Xiyuan Yang, Zhenyu Pan, Guo Ye, Han Liu
Abstract:
Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is completed. It does not explain whether the current behavior is making progress, remaining unchanged, or undoing earlier progress. For this reason, recent studies have increasingly explored progress rewards that provide feedback during task execution. However, the current literature lacks a shared framework. Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols. This makes it difficult to compare them and understand what their results actually validate. In this survey, we provide a unified view of progress reward modeling for robotic learning. We organize the field in three connected steps. We first study the interface of a progress model. This defines the problem from the outside by asking what information the model receives and what form of progress signal it produces. We then move inside the model and study the methods used to construct this signal. This reveals the different assumptions and mechanisms behind progress estimation and reward generation. Finally, we examine the data and benchmarks that support these methods. This shows how progress supervision is obtained and what different evaluations actually measure. Together, these three perspectives connect what a progress model is, how it is built, and how its quality is validated. We further summarize the main limitations of current approaches and discuss future research directions.
Authors:Jian Hu, Huiying Li, Hao Zhang, Binfeng Xu, Yifan Zhang, Shaokun Zhang, Hemil Desai, Michael Demoret, Pavlo Molchanov, Jan Kautz, Yi Dong
Abstract:
Agentic reinforcement learning research is constant algorithm modification, new estimators, new pipeline stages, new rollout schemes, and in mainstream frameworks each change threads through layers of trainer, distributed backend, and rollout glue: the cost lands on the researcher at every iteration. Molt is a PyTorch‑native training framework built to keep that cost small: a codebase compact and clean enough for a researcher to hold in their head, and for an AI coding assistant to read and reason about in its entirety, so the algorithm flow can be traced and changed end to end. The agent is an ordinary program, and one asynchronous loop trains multimodal and mixture‑of‑experts policies while never training on a token it did not generate, consistent in tokens, policy versions, and model semantics. Leanness does not cost performance: under a matched, fully asynchronous protocol, Molt is statistically comparable to a state‑of‑the‑art Megatron‑based stack. Molt is open source and provides recipes and containers at https://github.com/NVIDIA‑NeMo/labs‑molt.
Authors:Kaiwen Wang, Frank Bieder, Yinzhe Shen, Carlos Fernandez, Jan-Hendrik Pauls, Omer Sahin Tas
Abstract:
Simulation‑to‑reality translation must bridge the appearance gap between synthetic and real domains while preserving structural and semantic consistency. Conditioning‑based methods achieve spatial alignment but introduce computationally expensive control modules. Paired‑data methods achieve realism but rely on complex synthesis pipelines, often altering scene geometry and semantics. Training‑free editing methods avoid both constraints but lack a learned appearance prior, limiting their perceptual quality. Recently proposed phase‑preserving diffusion presents a promising alternative, but Fourier‑domain formulations are constrained by global spectral coupling. This coupling induces spatial artifacts such as ringing and boundary leakage, thereby degrading structural and semantic consistency. We introduce Wavelet Phase Diffusion, which addresses this through two components. First, we operate in the Dual‑Tree Complex Wavelet Packet Transform domain, whose localized wavelet packets enable spatially adaptive phase injection without global spectral interference. Second, Low‑Frequency Randomization (LFR) replaces the low‑frequency packet, decoupling the model from the synthetic illumination prior and enabling in‑distribution real‑world appearance. Both components train on unpaired open‑domain data, and introduce negligible inference overhead. The spatial locality further enables instance‑level translation, where individual objects or regions are translated to photorealistic appearance independently while the surrounding scene remains untranslated. On vKITTI \to KITTI image translation, ours outperforms prior methods in realism and semantic consistency while maintaining competitive structural alignment. For CARLA video translation, ours approaches the realism of paired‑data methods while reducing VLM planner ADE and FDE by 5.4% and 5.1%, respectively.
Authors:Martin Andrews
Abstract:
We prove that two canonical local synaptic learning rules, the potentiation arm of spike‑timing‑dependent plasticity (STDP^+) and homeostatic plasticity (instantiated here via flashlight granule‑cell‑like neurons), together can implement the exact gradient of a SIGReg‑like self‑supervised learning objective. The equivalence requires no gradient calculations, no global error signals, no weight transport, and no label information: the only inputs are pre‑ and post‑synaptic firing rates, local firing statistics, and the temporal contiguity of natural sensory streams. On a synthetic clustering task designed to probe whether class structure can be recovered from temporal ordering of inputs alone, ordered presentation raised cluster separation (CSR) to 2.49 while random ordering left it near baseline (0.83), a roughly threefold (\approx 3.5σ) separation attributable solely to input ordering. On temporally ordered MNIST, a two‑layer network trained entirely with these rules achieved 87.3% linear‑probe accuracy, showing that the mechanism functions end‑to‑end.
Authors:Bingjun Luo, Jialin Guo, Yue Yao, Xinpeng Ding
Abstract:
Multimodal Large Language Models (MLLMs) have achieved impressive performance, but their safety alignment remains vulnerable to jailbreak attacks. Existing content‑based jailbreaks are often inconsistent and show unsatisfying performance against the rapidly evolving MLLMs, failing to exploit non‑content‑based vulnerabilities. Unlike previous research, we empirically find that MLLMs exhibit a Stylistic Inconsistency between their comprehension ability and safety ability: MLLMs can robustly understand content regardless of visual style, yet their defense mechanisms can be easily bypassed by specific stylistic triggers. Based on this finding, we propose Adversarial Style Optimization (ASO), a plug‑and‑play enhancement module to amplify existing visual jailbreaks. ASO fine‑tunes an image‑editing model to superimpose an optimized stylistic modification onto a given adversarial image, using a Group Relative Policy Optimization (GRPO) agent guided by a Structurally‑Tiered Reward Function that combines a logit‑based signal for detecting explicit refusals with a high‑fidelity semantic evaluation from a powerful judge model. Extensive experiments show that ASO significantly enhances the ASR of SOTA attacks, demonstrating that stylistic biases are a scalable vector for red‑teaming MLLMs. Our code is available at https://github.com/bingjunluo/ASO.
Authors:Ashwath Vaithinathan Aravindan, Mayank Kejriwal
Abstract:
We investigate where and how transformer‑based language models commit to predictions in multiple‑choice question answering. We identify the _Hard Decision Layer_ (HDL), a natural architectural property where answer option rankings stabilize abruptly during inference. Empirical validation across four language models (Qwen, Llama, Granite, Mistral) and four benchmark datasets demonstrates consistent HDL emergence without learned routing policies. We also show that the HDL is invariant to fine‑tuning. Our results reveal striking accuracy improvements at the HDL: up to +0.61 (Qwen on CommonsenseQA), after which performance stabilizes. Systematic ablations on label formats and problem complexity confirm the phenomenon is fundamental to model architecture. These findings offer mechanistic insights into transformer inference and suggest opportunities for efficient reasoning and model steering. All code and results required to reproduce this work are available in https://github.com/Mystic‑Slice/hard‑decision‑layer
Authors:Miaobo Hu, Xiaobo Guo, Shuhao Hu, Bokun Wang, Rui Chen, Xin Wang, Daren Zha, Jun Xiao
Abstract:
Schema graphs are an upstream bottleneck of schema‑grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available. We introduce SCOPE (Schema Construction and Ontology‑induction Pipeline Evaluation), a train‑text‑only benchmark for corpus‑to‑schema induction and optional schema fusion from raw text, built from 24 public information extraction sources (15 RE and 9 EE) normalized into evaluation‑only gold schema graphs; its core event‑extraction target covers event types and within‑event argument roles, with inter‑event links reported separately. We present SCION (Schema Construction and Induction with Ontology Normalization), an auditable reference pipeline rather than a new extraction architecture; it constructs candidate spaces from train text and restricts naming, merging, filtering, validation, and conservative fusion to candidate‑linked evidence under strict JSON contracts. On the SCOPE core suite, SCION‑lite attains the highest F1 among released source‑schema references, Text2Onto‑style, LLM‑only, and matched extract‑then‑aggregate baselines under Literal, Fuzzy, Continuous, and Graph schema‑graph metrics, while the compact open‑model SCION‑RL variant reduces reliance on proprietary LLM schema engineers. These results are reported against normalized typed‑edge targets rather than as claims that induced schemas surpass human ontology design; the release includes evidence‑linked outputs, parse/fallback logs, candidate retention/merging logs, run manifests, code, and benchmark packages at https://github.com/wandugu/paper_scion.
Authors:Zeyu Ren, Ling Yue, Ran Li, Yishu Wang, Shengxiang Xu, Hanmo Liu, Shaowu Pan, Shimin Di
Abstract:
Large language model agents increasingly solve complex tasks by constructing inference‑time workflows that combine reasoning, tool use, and code execution. While such workflows enable flexible problem solving, the useful procedures discovered during execution are often transient: they help solve the current task but are not retained in a form that can systematically benefit future tasks. We present FlowEvo, a training‑free framework that compiles successful traces into reusable skill records. Each record pairs a callable artifact with auxiliary structured guidance, and admission applies interface, replay, and safety checks where feasible. These skill records persist in a skill bank at inference time. FlowEvo is organized around three coupled mechanisms: (1)~workflow‑to‑skill compilation, which extracts reusable executable artifacts from successful traces; (2)~skill‑to‑workflow feedback, which retrieves accumulated skills to support future problem solving through either direct execution or structured context injection; and (3)~skill curation, which monitors downstream utility and suppresses skills that cause negative transfer. Through this workflow‑‑skill‑‑workflow feedback loop, FlowEvo enables agents to accumulate and refine task‑solving capability over time without updating model parameters. Experiments on benchmarks spanning interactive environments (ALFWorld) and code/math generation (HumanEval, GSM8K) show that FlowEvo achieves the best accuracy‑cost tradeoff among the evaluated baselines under our implementation settings. On ALFWorld, FlowEvo achieves an 82.8% success rate, 23.6 percentage points above the strongest baseline, while its average token usage per episode is less than half that of the most efficient baseline. Controlled ablations confirm that each mechanism contributes to the overall result. The code is public at https://github.com/DEFENSE‑SEU/FlowEvo.
Authors:Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Ran Xu, Shijian Lu, Gongjie Zhang
Abstract:
Despite rapid progress, most existing vision‑language models (VLMs) built from 2D visual inputs often struggle when handling various 3D tasks that require fine‑grained spatial understanding and reasoning. To bridge this gap, we present VLM‑IE3D, a unified framework that enhances the 3D spatial awareness of VLMs by equipping them with both implicit and explicit 3D geometries learned from RGB videos. Our VLM‑IE3D introduces Implicit Geometry Tokens (IGTs) that capture high‑level geometric priors from input videos, as well as complementary Explicit Geometry Tokens (EGTs) that encode detailed geometric structures from reconstructed 3D attributes. On top of that, VLM‑IE3D comes with a 3D‑aware adapter that effectively fuses the two types of geometric representations with 2D visual cues. This RGB‑only design injects strong 3D inductive biases for fine‑grained spatial understanding and reasoning without requiring any additional 3D inputs. Extensive experiments show that VLM‑IE3D achieves superior performance consistently across various 3D tasks including 3D video detection, 3D visual grounding, 3D dense captioning, and spatial reasoning. Code and models are available at https://github.com/Vegetebird/VLM‑IE3D.
Authors:Sicheng Mo, Yuheng Li, Ziyang Leng, Krishna Kumar Singh, Bolei Zhou
Abstract:
Multi‑agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi‑agent and multi‑view settings. We present WorldWeaver (W^2), a streaming multi‑agent video diffusion model that augments rollout with cross‑agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual agent status, global state views including bird's‑eye views, and scene text. We further improve the architecture with a Mixture‑of‑Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two‑agent Minecraft video generation show that explicit world‑state modeling improves logical consistency and generation quality.
Authors:Yihong Sun, Seoung Wug Oh, Jiahui Huang, Bharath Hariharan, Joon-Young Lee
Abstract:
Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task‑specific annotations are fragmented across incompatible, domain‑specific datasets. Current unified systems circumvent this by restricting training to fully co‑annotated data, or by incurring the large computational cost of pseudo‑labeling. To mitigate this, we introduce UniD, a unified video model that jointly predicts eight dense scene properties‑depth, surface normals, semantic segmentation, boundaries, human parts, albedo, shading, and materials‑all learned from disjoint, domain‑specific datasets. We propose a simple yet effective distillation step in which per‑task experts supervise a unified backbone through lightweight task projectors, eliminating the need for annotation overlap or pseudo‑labeling. Our key insight is that the strong visual priors of a pretrained diffusion model are sufficient to bridge the domain gaps introduced by disjoint training sources, enabling robust generalization to scene‑task combinations never seen during training. UniD achieves competitive performance against per‑task specialists and multi‑task baselines, with strong generalization to out‑of‑distribution scenarios and enhanced temporal and cross‑task consistency. Code and video results are available at https://unid‑video.github.io/.
Authors:Rogerio Guimaraes, Pietro Perona
Abstract:
Diffusion and flow‑matching models dominate conditional image generation, yet inference‑time scaling for these models is far less developed than for autoregressive language models. Because final quality is highly sensitive to the initial noise seed, many approaches spend extra compute on seed search or resampling under a black‑box reward, but typically maintaining a constant memory footprint throughout inference. We show that relaxing this constraint enables an underexplored inference‑time scaling axis: by front‑loading exploration, evaluating many seeds early, and pruning aggressively, we can use a fixed compute budget more effectively. \emphProgressive Seed Pruning (\PSP) scores intermediate denoised estimates and progressively narrows the candidate set so that only promising trajectories are fully denoised, while keeping the total number of model evaluations fixed. Across diffusion and flow‑matching backbones, \PSP \ consistently improves reward‑guided selection and achieves higher GenEval scores (automated) and better human evaluation on prompt‑alignment than best‑of‑N, importance‑sampling, and tree‑search baselines at matched compute. Project page: https://www.vision.caltech.edu/psp. Code: https://github.com/rogerioagjr/psp.
Authors:Mengfei Zhao, Dihong Huang, Yikai Tang, Peihao Li, Mingxuan Yan, Ruiqi Zhuang, Yanjia Huang, Jie Wang, Hai Zhai, Tony Zhou, Rui Zhang, Zhexi Luo, Yuchen Huang, Jianfei Yang, Jiachen Li
Abstract:
Learning effective robot manipulation policies requires diverse, high‑quality demonstrations, yet existing data pipelines are often difficult to scale because they rely on specialized hardware, centralized operators, or fixed task suites. We present AXIS, a growable community‑driven data engine and benchmark for scalable robot learning, which enables browser‑based teleoperation for large‑scale demonstration collection, automatically generates and validates new manipulation tasks, and transforms community‑collected demonstrations into training‑ready data through automated success checking, quality filtering, trajectory smoothing, and visual and physics‑based augmentation. The AXIS dataset currently contains 207 diverse tasks and 50K+ trajectories. Meanwhile, AXIS organizes data into task snapshots and evaluates policies with a systematic held‑out protocol. We compare vision‑language‑action (VLA) policies under a unified AXIS evaluation suite and analyze scaling behavior across different data volumes. Continual pretraining on AXIS substantially improves the overall success rate of π_0.5 by 5.8%, outperforms the model pretrained on RoboCasa365 by 37.3%, and exhibits consistent scaling with increasing data volume, with the largest gains observed under layout, sensor‑noise, and camera perturbations.
Authors:Lukas Knobel, Andrew Zisserman, Yuki M. Asano
Abstract:
Understanding motion in video is a fundamental challenge for visual learning, as frame‑to‑frame change entangles two sources of dynamics: camera motion and object motion. This decomposition has remained underexplored in representation learning, partly because these factors are tightly coupled in natural videos and difficult to supervise separately. Yet recovering it is important for learning robust motion representations that separate meaningful object dynamics from camera‑induced variation. We study whether such structured motion representations can be recovered from frozen features of a pretrained image vision transformer. We propose the Structured Dynamics Model (SDM), which explicitly separates the dominant source of temporal change from residual dynamics through future‑feature prediction, rather than representing video change with a single entangled latent or with unstructured, spatially dense transition tokens. Training combines self‑supervised learning on real video with weak supervision of scene dynamics on synthetic Kubric data. We evaluate SDM on ProbeMotion, a new evaluation suite spanning synthetic and real videos with camera motion, object motion, and combined dynamics. SDM outperforms backbone baselines using global CLS or average‑pooled features, and compares favorably to strongly supervised representations such as VGGT on several probes, despite using substantially weaker supervision. These results suggest that pretrained image models can be readily repurposed into structured video‑dynamics representations, providing a useful inductive bias for learning and analyzing latent video dynamics.
Authors:Mengshi Qi, Xiaoyang Bi, Xianlin Zhang, Huadong Ma
Abstract:
Self‑supervised depth estimation is challenging for safe autonomous driving under various adverse weather conditions due to sensor perception degradation. These challenges arise from two main aspects. Firstly, adverse conditions can distort pixel correspondences and violate the assumptions embedded in the self‑supervised loss function, leading to erroneous depth predictions. Secondly, while radar is a widely adopted sensor in adverse weather conditions, the sparse distribution of radar points in the Point of View (POV) poses challenges for self‑supervised fusion. To address these issues, we introduce a novel self‑training pipeline using unpaired real all‑weather data through multi‑teacher distillation and robust radar fusion. We propose the Uncertainty‑Aware Multi‑Teacher Distillation method to generate diverse teacher models with different adverse condition inputs, and then employ uncertainty modeling to weigh the knowledge distillation loss. Additionally, we design the POV‑BEV Radar Fusion approach, which leverages camera‑pixel ray constraints to establish connections between the camera's Point of View (POV) and the radar's Bird's‑Eye View (BEV). This approach enables the utilization of denser radar points, effectively capturing the complementary perspectives of both POV and BEV. Extensive quantitative and qualitative experiments demonstrate the robustness of our proposed method on all‑weather datasets, achieving state‑of‑the‑art performance. Our code and models are available at https://github.com/MICLAB‑BUPT/RobustDepth.
Authors:Hongxin Zhang, Chunru Lin, Junyan Li, Zhou Xian, Tsun-Hsuan Wang, Chuang Gan
Abstract:
Creating dynamic and physically realistic 4D worlds from natural language descriptions is both fascinating and challenging. Traditional computer graphics methods rely on manual creation, requiring extensive human effort to fine‑tune materials, motions, and visual fidelity. Recent advances in generative foundation models have sparked interest in learning to generate such 4D worlds from large‑scale data; however, existing methods still struggle to ensure physical plausibility and controllability. In this work, we take a different path by leveraging foundation models to construct an agentic system that emulates how humans traditionally create 4D worlds, yet automates the entire process. We present GS‑Agent, an end‑to‑end multi‑agent framework that integrates physics engines in the loop to generate realistic, dynamic, and controllable 4D physical worlds from natural language. Inspired by how humans build 4D worlds, GS‑Agent decomposes the task into entity management, covering 3D asset curation, material tuning, placement, and motion control, and rendering configuration, including camera and lighting manipulation. Multiple agents with distinct expertise interact with the physics engine via code, seek multimodal feedback, and collaborate to iteratively construct 4D worlds that align with the given descriptions. Experimental results show that GS‑Agent effectively converts natural language into diverse and physically plausible 4D worlds exhibiting rich interactions among liquids, deformable objects, and rigid bodies, while achieving cinematic camera and lighting control. We envision GS‑Agent as a foundation for a new paradigm in 4D world generation, empowering creative content creation and physical AI. Project page at https://umass‑embodied‑agi.github.io/gs‑agent/
Authors:Piotr Wilam
Abstract:
Do independently trained language models come to represent the same thing in the same way? We answer for code, extending a recently introduced concept‑circuit extraction method to a 2x2 design ‑‑ Python and Rust crossed with Qwen2.5‑Coder‑7B and DeepSeek‑Coder‑V1‑6.7B ‑‑ and measuring a complete inventory of grammatical concepts (58 Python, 57 Rust) identically in all four cells: the smallest design that separates what depends on the task, the language, and the model. The answer splits into three parts. What earns dedicated circuitry is set by the task: the models agree on which concepts receive circuits (Spearman ρ = 0.638 for Python, 0.673 for Rust, both p < 10^‑7). Where those circuits sit is set by the model: Qwen processes concepts in a late band (~L17‑19), DeepSeek at L6‑7, for both languages. How circuits grow across layers is also set by the model: Qwen gives its atomic concepts an early spike that DeepSeek does not. "Are circuits universal?" thus has no single answer: yes for What, no for Where and How ‑‑ universality is a property of representational content, not of computational organisation. None of this structure was fixed in advance. The agreement could have landed anywhere between independence and identity; it lands at ρ\approx 0.65. Rust constructs receive 2‑3x more concept‑specific circuitry than their Python equivalents, in both models. Both models share neurons between the languages (6/7 and 7/7 paired constructs), DeepSeek 1.94x more than Qwen ‑‑ a direction no prior result predicts. And Qwen binds nine keywords of Rust's type‑and‑trait machinery into one tight neuron cluster (Jaccard 0.535 vs null 0.112, p < 0.001), a semantic dimension invisible in surface syntax. Ablation and linear probes confirm the circuits are functional. All claims are scoped to this 2x2; whether the per‑model profile predicts a third model is the designed next test.
Authors:Mack Nixon, Liam Wright, Yevgeniya Kovalchuk, Alison Fang-Wei Wu, Martin Danka, Andy Boyd, David Bann
Abstract:
Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third‑party cloud‑based models. Their adoption in research using personal data is constrained by governance requirements that typically prohibit data transmission to external services. Locally deployable open‑weight models offer an alternative since sensitive data never leave the local environment. We introduce an open‑source framework for evaluating the efficacy of AI agents powered by open‑weight LLMs on one of the most persistent bottlenecks in research on longitudinal population studies: data preparation. The framework comprises: a curated ground‑truth dataset (cleaning scripts preparing six sweeps of data from a British cohort study), task definitions encompassing tasks such as category harmonization and multi‑wave merging, and automated routines for evaluating the LLM‑produced R code and outputted data. We benchmark LLMs across the (consumer grade) deployment spectrum to assess their efficacy in 20 data preparation tasks (creation of 102 variables). Current state‑of‑the‑art, 31‑35B parameter models almost saturated our benchmark ('average task completion' up to 87.9%). The performance of open‑weight LLMs running on consumer‑grade hardware shows promise of a viable path toward AI‑assisted data preparation in governance‑restricted research settings. Our framework is publicly available at: https://github.com/UCL‑ARC/RRBench.
Authors:Yukun Shi, Minglun Gong
Abstract:
Dynamic‑scene reconstruction is almost always evaluated inside the observed time window, yet deployment settings such as AR overlays, robot interaction, and anticipatory planning need the future surface: the geometry at times beyond those captured. No standard benchmark measures this. We introduce FutureSurf, a controlled diagnostic benchmark and dataset for future‑time surface reconstruction that trades scene diversity for exact future ground truth and falsification controls. A method trains on the observed first 75% of a sequence; we score its extracted per‑frame surface on the held‑out future by Chamfer distance, reporting absolute future CD as the primary score and the future/observed gap as a diagnostic. The dataset contains eight analytically defined controlled motions, including three falsification controls, with exact per‑frame ground‑truth meshes. We also provide a ground‑truth‑side recoverability oracle. The release includes split files, scoring code, a benchmark card, and Croissant metadata. On the controlled motions, the DG‑Mesh backbone leaves a 2.7‑4.1× gap even for futures predictable in principle (four of five recoverable from observed motion by a fixed rule), while the falsification controls behave as designed (the surface‑invariant motion shows no gap). Beyond the contributed dataset, the gap persists across six animated DG‑Mesh asset scenes and a second backbone, Deformable‑3DGS (2.0‑6.6×; both share a deformation‑MLP temporal model). The benchmark also shows that future rendering quality and future‑surface accuracy are statistically decoupled, so the novel‑view‑synthesis metrics the field reports do not track future geometry. The future error is structured, concentrating where the surface moves. The dataset, evaluation toolkit, and scoring code are available on Hugging Face and GitHub (https://github.com/Ricky‑S/futuresurf).
Authors:Kui Jiang, Zefan Feng, Laibin Chang, Yan Luo, Junjun Jiang, Xiaopeng Fan
Abstract:
Underwater image enhancement remains challenging due to wavelength‑dependent light absorption, scattering, and backscattering, which jointly cause color distortion, contrast degradation, and detail loss. Since these degradations vary with scene depth and imaging conditions, different regions within the same image often exhibit heterogeneous degradation patterns and thus require region‑adaptive restoration. Although visual RWKV models offer an efficient linear‑complexity solution for long‑range dependency modeling, their predefined scanning orders are content‑agnostic and therefore fail to adapt recurrent state propagation to spatially non‑uniform restoration demands. To address this limitation, we propose a Clustering‑aware RWKV framework, termed CRWKV, which reformulates the fixed recurrent propagation path of conventional RWKV into a content‑adaptive token trajectory. Specifically, we introduce Clustering‑aware Semantic Dynamic Reordering (CSDR), which groups tokens according to semantic feature similarity and derives a dynamic traversal order from inter‑cluster contextual relations. This design enables WKV states to be accumulated along semantically correlated regions rather than fixed spatial or spectral orders. Since dynamic reordering may disrupt the local continuity of original spatial neighborhoods, we further propose Dark‑response Modulated Local Propagation (DMLP), which extracts local structural responses via depth‑wise convolution and adaptively modulates their propagation strength using a neighborhood‑aware pseudo‑dark response map. In this way, local structural cues are compensated before recurrent aggregation while preserving content‑adaptive long‑range modeling. Extensive experiments on multiple underwater image enhancement benchmarks demonstrate that CRWKV achieves state‑of‑the‑art quantitative performance and superior visual quality.
Authors:Zhongming Liu, Bingbing Jiang, Guangxin Wan, Xiang Zou
Abstract:
Steel surface defect segmentation is critical for industrial quality inspection, yet existing methods struggle with elongated, anisotropic defects such as cracks and scratches due to the isotropic receptive fields of standard convolutions and rigid sampling grids that cannot adapt to irregular defect boundaries. To address these limitations, we propose Strip‑based Predictor for Deformable Convolutional Networks (SPDCN) with two key innovations. The Fuzzy‑enhanced Multi‑scale Context Module (FMCM) employs group‑wise multi‑branch convolutions with an intuitionistic fuzzy channel attention mechanism to adaptively capture multi‑scale contextual information across varying defect sizes. The Adaptive Direction‑Aware Deformable Convolution (ADADC) replaces the conventional offset predictor with decoupled horizontal and vertical strip convolutions, enabling the deformable sampling grid to anisotropically align with the principal orientation of elongated defects. Extensive experiments on public steel surface defect benchmarks demonstrate that SPDCN consistently outperforms state‑of‑the‑art methods, achieving 89.60% mIoU on NEU‑Seg with only 3.54M parameters. The source code is publicly available at https://github.com/DWlzm .
Authors:Tong Ling, Wenhui Diao, Yingchao Feng, Hanbo Bi, Zhongyan Hou, Xian Sun
Abstract:
Monocular depth estimation is a fundamental prerequisite for 3D reconstruction and autonomous navigation in Unmanned Aerial Vehicles (UAVs). In practical deployments, UAVs operate under highly dynamic camera poses characterized by continuous variations in height, pitch, roll, and field of view (FOV). Existing monocular depth estimation methods frequently fail to generalize across such diverse perspectives and the expansive scale of depth distributions inherent in aerial scenes. To address these challenges, we establish a quantitative representation of UAV viewing angles through rigorous theoretical analysis, deriving the geometric correspondence between viewing angles and view distances using the ground plane as a reference for observation. Building upon this, we propose Depth Estimation for Any Perspectives Model (DAPM), representing the first monocular framework specifically designed for UAV aerial imagery to jointly estimate camera pose and depth under continuously varying viewpoints. Specifically, we introduce an Ideal Ground Depth (IGD) module that leverages the derived geometric relationships between UAV perspectives and view distances to implement dense camera‑pose supervision and enhance depth features. And we further develop a coarse‑to‑fine Progressive Quantization Bins (PQB) module. By incorporating progressive supervision and hierarchical quantization bins, the PQB module enables robust estimation in complex UAV aerial imagery. To evaluate the proposed framework, we present the UAV Any Perspectives Depth (UAPD) dataset, featuring comprehensive and continuous distributions of pose parameters. Experimental results on UAPD demonstrate that DAPM achieves state‑of‑the‑art performance across both depth and camera‑pose estimation metrics. The source code and datasets are available at: https://github.com/ThisIsLT/DAPM.
Authors:Panagiotis Mermigkas, Argyris Manetas, Petros Maragos
Abstract:
Existing Gaussian‑splatting‑based monocular Simultaneous Localization and Mapping (SLAM) systems are either tailored to short sequences, are not real‑time, or suffer from prohibitive GPU memory requirements, limiting their applicability in realistic, long‑horizon scenarios. To address this, we present GLAM‑SLAM, a real‑time, decoupled Gaussian‑splatting SLAM system designed for large‑scale outdoor scenes. We ensure lightweight tracking using a robust, feature‑based SLAM frontend, while for mapping, we adopt a structured, sparse anchor grid representation that ensures scalable operation and maintains scene coherence across long‑term sequences. To satisfy the dense initialization requirements of 3D Gaussian Splatting (3DGS), we introduce a geometry‑based flow‑densification anchoring strategy using epipolar constraints. Furthermore, by treating mapping as a multi‑scene problem, we propose a scene‑partitioning strategy that introduces a strong spatial inductive bias via MLP initializations to generate localized Gaussians. We evaluate our system on the challenging, long‑sequence KITTI Odometry, Oxford RobotCar, and M'alaga datasets. Extensive ablations and comparisons demonstrate a 15% improvement in reconstruction quality over the second‑best performer, while maintaining real‑time performance and the ability to scale to longer sequences. Code is publicly available for the benefit of the community.
Authors:Dongbin Na
Abstract:
A vision‑language AI assistant returns its answer as a stream of generated tokens. Therefore, a safety guard that watches that answer has to keep up with the stream and stop a harmful reply before a user reads it. Recent vision‑language guardrails instead generate a chain of thought before they issue a verdict. They believe that step‑by‑step reasoning yields a safer guard. This design makes the guard heavy and slow, since the model must decode many tokens for harmfulness detection. We pose the question of whether a vision‑language guard really needs to reason in order to screen a response. We answer with a guard that has no chain. ResponseGuard reads a harmful verdict from a single pooled representation of the request, the response, and the image in one forward pass. Across a standard multimodal guardrail benchmark, our 2B ResponseGuard outperforms a recent 3B reasoning‑based vision‑language guard on response harmfulness detection, without any reasoning and at about 150 times lower time cost. On request harmfulness the reasoning guard retains an overall lead, and the remaining gap on both tracks sits on the image‑only cells. We observe that the gap may stem from the frozen vision encoders that both designs use rather than from the missing chain. We have also found the reasoning guard directs almost none of its verdict attention to the image. Based on a single‑pass detection, ResponseGuard can screen an answer sentence by sentence as it streams and stop a harmful answer before it finishes. For guarding the response of a vision‑language model, a calibrated single‑pass label may provide a sufficient safety signal. We fully release all source code, trained models, and datasets at https://github.com/ndb796/ResponseGuard.
Authors:Jiabin Lou, Haopeng Wang, Yuanshuai Wang, Xinyu Liu, Xuxin Lv, Yuxin Guo, Lei Huang, Rongye Shi, Wenjun Wu
Abstract:
Vision‑and‑Language Navigation (VLN) enables embodied agents to follow natural‑language instructions. However, route‑level instructions commonly encode spatial priors, such as orientation, distance, and layout, that are not explicitly available from onboard sensing at deployment in open, GPS‑denied environments. Benchmark performance under such interfaces therefore jointly reflects visual navigation ability and the use of route structure explicitly supplied by the task description. As a complementary formulation, we propose Vision‑Only Long‑Horizon Navigation (VoLN), which shifts route‑relevant information from externally supplied instructions and global guidance to locally observable in‑scene cues. In VoLN, goal views specify the destination, while route‑relevant information is available only through locally observable in‑scene cues that the agent must detect, interpret, and select online. We instantiate VoLN for aerial navigation through VoLN‑UAV, a 7,210‑episode benchmark that combines long‑horizon goal‑directed flight, continuous 3D motion, large viewpoint changes, and context‑dependent beacon selection. We further provide VoLN‑MLLM as an initial reference baseline. It aligns self‑supervised visual features with a structured semantic space and predicts short‑horizon waypoint segments from observation history, goal views, retrieved visual‑‑semantic tokens, and proprioception. On the five‑environment Test‑Unseen split, it obtains success rates of 7.4%, 4.5%, and 1.8% on Easy, Normal, and Hard episodes, respectively. These results provide an initial evaluation of VoLN and reveal substantial remaining challenges in long‑horizon evidence integration, cross‑view goal matching, and closed‑loop stability. Project page: https://admire‑ljb.github.io/VoLN‑UAV/
Authors:Alan Edelman, Timothy E. Holy
Abstract:
Positive‑semidefinite matrices are most efficiently factored using the Cholesky decomposition. For indefinite matrices, the Cholesky factorization does not exist, and the alternatives face greater challenges in achieving numeric stability and preservation of banded structure. Here we pursue an analogy between the requirement for positive‑semidefinite matrices and the solution of the quadratic equation x^2 = c for c <= 0. It is shown that a non‑associative algebra, called the hemiplex numbers, allows the Cholesky factorization to be computed for arbitrary symmetric matrices. Crucially, the hemiplex Cholesky factorization does not require pivoting for its existence or stability, allowing it to preserve banded structure. For singular matrices it produces a parametrization of the null space, and provides opportunity for truncation of nearly‑null directions in a manner similar to common usage of the singular value decomposition. The hemiplex Cholesky factorization may be a practically useful addition to the tools for solving symmetric linear equations.
Authors:Sung-Hoon Yoon, Hoyong Kwon, Changgyoon Oh, Kuk-Jin Yoon
Abstract:
Open‑vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories. While the self‑supervised model DINOv3 provides strong structured visual representations, its lack of native textual alignment hinders its direct application to OVSS. To bridge this gap, we propose DINOde, an ODE‑based framework that continuously aligns CLIP text embeddings with the DINO visual manifold. Our approach employs two complementary components: (i) Semantic Text Flow (STF), which evolves text embeddings toward the DINO manifold through a continuous ODE trajectory, and (ii) Global Context Flow (GCF), which progressively refines the holistic image representation carried by DINO's CLS token. To preserve the hyperspherical geometry of the feature space during this evolution, we further introduce Velocity Tangent Projection, which constrains the learned velocity field to the tangent space. By modeling alignment as a continuous trajectory, DINOde avoids the manifold entanglement inherent in discrete MLP projections and yields more robust cross‑modal alignment. Extensive experiments demonstrate that DINOde consistently outperforms existing methods and achieves state‑of‑the‑art performance across multiple OVSS benchmarks. The code is available at https://github.com/yoon307/DINOde.
Authors:Wenbin Duan, Yan Shu, Zhuoyuan Fu, Fangmin Zhao, Yan Li, Yaru Zhao, Binyang Li
Abstract:
Rectified‑flow‑based diffusion transformers, particularly FLUX, have demonstrated outstanding performance in high‑quality image generation. However, achieving fast and accurate inversion‑‑transforming images back to latent noise for faithful reconstruction and editing‑‑remains a challenging bottleneck due to the discretization errors of linear solvers. This paper introduces SlerpFlow, a straightforward yet highly effective zero‑shot approach that unlocks the full potential of FLUX for high‑fidelity inversion and editing. Unlike existing approaches (e.g., RF‑Solver) that rely on complex numerical approximations such as high‑order Taylor expansions to correct trajectory errors, we present a geometric view based on the Manifold Hypothesis: the empirically observed trajectory curvature is not a numerical artifact, but rather serves as a necessary "centripetal force" that constrains the flow to remain on the data manifold. Guided by this insight, SlerpFlow integrates Spherical Linear Interpolation (Slerp) to rectify flow velocity directions on the hypersphere, strictly adhering to the intrinsic curvature of the latent space. Crucially, by caching the corrected velocity for subsequent steps, SlerpFlow achieves high‑precision inversion while maintaining the computational efficiency of a first‑order Euler solver. Extensive experiments on FLUX‑based reconstruction and editing tasks demonstrate that SlerpFlow improves reconstruction fidelity and achieves stronger semantic alignment in editing without requiring additional training. Code is available at https://github.com/0answer0/SlerpFlow.
Authors:Siyu Li, Kunyu Peng, Di Wen, Beiping Hou, Zhiyong Li, Kailun Yang
Abstract:
Topological maps are key outputs of autonomous driving perception systems, delivering essential road information for path planning. They identify instances such as centerlines and traffic signs, along with their connectivity relationships. Due to the lack of explicit markings for centerlines in real‑world environments, the detection of centerline instances remains a significant challenge. To tackle this problem, we propose HGeo‑TopoMap, which leverages an explicit prior map and implicit spatial relations to hierarchically boost topological mapping. First, a geometric adaptive learning module is designed for the road structure map obtained via inverse perspective mapping. This module discretely encodes semantic and spatial features from the map, followed by a prior‑mask attention mechanism that selectively focuses on informative regions. Then, a geometric consistency learning module is devised, which leverages the geometric properties and spatial relationships of centerlines. Built on the geometry‑aware decoder, it enforces spatial consistency by aligning features of centerline instances with identical geometric orientations. The proposed method is evaluated on the OpenLane‑V2 dataset across the centerline, lane segment, and robustness benchmarks. Beyond substantial improvements in topological mapping accuracy, the proposed method offers the benefit of enhanced robustness, consistently outperforming baselines under both standard and challenging conditions. The source code and model weights will be made publicly available at https://github.com/lynn‑yu/HGeo‑TopoMap.
Authors:Zhongchen Zhao, Jixin Wang, Qi Xie, Hui Lin, Lei Zhang, Deyu Meng, Zongben Xu
Abstract:
Equivariant networks embed geometric symmetries as structural priors through weight sharing, achieving remarkable parameter efficiency across vision tasks. However, this parameter efficiency does not translate into compute efficiency: existing implementations unroll the structured weights into dense matrices and dispatch them to generic dense kernels, so the FLOPs of an equivariant layer are no smaller than those of a non‑equivariant counterpart. In this paper, we observe that the equivariant linear (EQ‑Linear) layer‑‑‑the most fundamental and frequently used module in modern equivariant architectures‑‑‑is essentially a circular convolution along the group dimension composed with a linear transform along the channel dimension. Building on this observation, we propose Flash EQ‑Linear, an exact acceleration algorithm that reduces the complexity from \mathcalO(NDC) to \mathcalO(NDC/T) by combining the Fourier convolution theorem along the group dimension with the conjugate symmetry of the real DFT. We further provide dedicated CUDA kernels for Flash EQ‑Linear, covering both forward and backward passes and both FP32 and FP16 precision. At the operator level, Flash EQ‑Linear achieves up to 2× forward speedup over PyTorch's F.linear; at the network level, Flash EQ‑ViT and Flash EQ‑Swin achieve up to 1.7× end‑to‑end speedup over both equivariant and non‑equivariant baselines. To our knowledge, this is the first time equivariant networks strictly dominate their non‑equivariant counterparts along all three axes simultaneously: accuracy, parameter efficiency, and inference speed.Code is available at https://github.com/zhongchenzhao/FlashEQLinear.
Authors:Chen Zhu, Xiaolu Wang, Weilong Zhang
Abstract:
In many social‑science research tasks, such as economics, LLM‑based agents must produce outputs for which no cheap, task‑complete, machine‑readable correctness signal exists. This creates a distinctive reliability problem for multi‑agent systems: how should generation, critique, coordination, and human judgment be organized when no component can certify the final result? We address this problem through pAI‑Econ‑claude, a gated, human‑in‑the‑loop multi‑agent architecture for AI‑assisted economic theory development. Agents coordinate through a shared workspace of inspectable intermediate records; specialized gates diagnose targeted failure modes and recommend loopbacks without certifying correctness; and human checkpoints retain authority over decisions that are costly to reverse. We evaluate the architecture on five matched economic‑theory tasks against an ungated baseline. Two evaluators blinded to configuration agreed on all five pairwise rankings, preferring the gated architecture in four tasks and the baseline in one. Mean failure severity fell from 1.58 to 1.16, while overall usefulness rose from 2.60 to 3.10. The largest observed gain occurred when a reality check rejected a false market‑structure premise and a proof review prompted revision of a false welfare claim. The negative case shows that scaffolding can also compress an economically important mechanism too aggressively. The results support a bounded claim: gated oversight improves the auditability of AI‑assisted economic theory without substituting for formal verification, and the allocation of irreversible human judgment is a more informative design variable than pure agent autonomy. The workflow is publicly available at https://github.com/maxwell2732/pAI‑Econ‑claude.
Authors:Linlin Wang, Xue Yang, Zhihuang Zhou, Zhenyu Zhong, Ruiyuan Zhang, Yansheng Li
Abstract:
Structured understanding of satellite video is essential for advancing dynamic geospatial scene analysis from low‑level perception to high‑level cognition. To move beyond object‑centric perception, this paper introduces spatio‑temporal panoptic scene graph generation (TPSG) in satellite video as a new benchmark task. TPSG aims to generate a structured graph composed of a set of triplets <subject, relationship, object> with explicit temporal spans, thereby describing dynamic geospatial scenes by jointly modeling identity‑consistent instance masks and spatio‑temporal relationships among panoptic scene elements. However, there is still no dedicated dataset for TPSG in satellite video. Moreover, TPSG in satellite video is intrinsically challenging, as objects are often small and weakly textured, cross‑frame association is easily disrupted by occlusion and background clutter, and relationship semantics are highly coupled with spatial structure and temporal evolution. Consequently, TPSG models developed for natural videos are not directly applicable to satellite video. This paper presents T‑STAR, a large‑scale benchmark dataset for TPSG in satellite video, comprising over 1.1 million instance masks and over 3.8 million spatio‑temporal triplets across 39 fine‑grained object categories and 70 fine‑grained relationship categories. To enable TPSG in satellite video, we propose a unified framework to enhance cross‑frame instance consistency and spatio‑temporal relationship prediction. Extensive experiments demonstrate the significance of T‑STAR and the effectiveness of the proposed framework, establishing a strong benchmark for future research on structured satellite video understanding. The dataset and code are available at https://github.com/linlin‑dev/T‑STAR.
Authors:Rafaello Sanna, William E. Byrd, Nada Amin
Abstract:
We present chrKanren, a dialect of the purely relational constraint logic programming language miniKanren which includes support for Constraint Handling Rules (CHR), a language for writing rule‑based programs such as constraint solvers. We show how to integrate CHR's constraint propagation mechanism into the language of miniKanren search streams such that both processes remain complete. We also use chrKanren to illustrate novel applications of constraints in miniKanren, such as semantic unification of user‑defined data structures and type‑and‑example‑directed synthesis for relational interpreters in the style of MYTH.
Authors:Chufeng Jiang, Neng-Fa Zhou
Abstract:
The Single Constant Multiplication problem is a fundamental NP‑hard optimization task in hardware design, which seeks to decompose a fixed constant using only additions, subtractions, and bit‑shifts. Although dynamic programming methods can produce near‑optimal SAT encodings for SCM, their encoding cost remains high for large constants. We propose a neuro‑symbolic framework that accelerates SCM SAT encoding by identifying good rules for guiding operator selection during decomposition. Our approach employs a graph neural network model to predict promising operator types from constant decompositions, and exploits the resulting confidence scores to prune no‑good choices in the symbolic search. Experimental results on unseen 17‑32 bit constants demonstrate one to two orders of magnitude reductions in encoding time, over 97% reduction in memory usage, and an order‑of‑magnitude decrease in branching, while preserving near‑optimal encoding quality in terms of additions. These results show that learning‑guided symbolic strategies can significantly improve the scalability and efficiency of SCM encoding. Our code and data are publicly available at: https://github.com/Chufeng‑Jiang/SCM_MLDP
Authors:Yunpeng Hua, Hongwei Yu, Jiawei Li, Qiankun Liu, Huimin Ma, Jiansheng Chen
Abstract:
Infrared image super‑resolution (IISR) mitigates the limitations imposed by low spatial resolution. Existing methods have recognized that IISR should preserve consistency in global distribution and structural information while enhancing image clarity. However, these methods are either insufficient or overly intrusive, a problem that becomes even more pronounced in diffusion‑based models. To address these issues, we propose a dual‑path diffusion‑based framework for IISR, termed Shift‑IISR. The proposed method is designed to improve the consistency of IISR results while preserving the generative capacity of diffusion models. Specifically, we develop a Global Representation Modulation (GRM) module to extract modality‑specific information from infrared imagery and guide the global distribution of the diffusion model toward the ground truth. In addition, we introduce a Local Structure Refinement (LSR) module to encourage the model to focus on structural information at each step of the iterative denoising process. Extensive experiments demonstrate that the proposed method effectively improves distributional and structural consistency while maintaining competitive super‑resolution performance. The source code of the proposed Shift‑IISR can be available at https://github.com/Assassink8/Shift‑IISR.
Authors:Armin Attarzadeh, Mohammad Ali Ghaemifar, Alireza Khanzadeh, Soheil Ganjefar
Abstract:
Bilateral teleoperation systems that include joint flexibility better reflect real robotic systems used in surgery, space, and rehabilitation. However, joint flexibility together with time‑varying communication delays makes it difficult to maintain stable and coordinated motion between the master and slave robots. To address this, we propose a hybrid control method that combines a stable Proportional‑plus‑Damping (P+d) controller with a model‑free deep reinforcement learning agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The P+d controller provides basic stability under bounded delays, while the learning agent adjusts and tunes the remote‑side proportional and damping gains in real time to reduce vibrations and improve tracking. Stability is guaranteed for bounded time‑varying delays using Lyapunov‑Krasovskii analysis. The approach provides a practical solution for teleoperation systems facing both joint flexibility and uncertain network delays.
Authors:Hariharan Ramesh, Jyotikrishna Dass
Abstract:
Fine‑tuning Vision Transformers (ViTs) with low‑rank adapters (LoRA) promises better communication efficiency under federated setup, yet existing aggregation strategies face fundamental limitations. Independently averaging these LoRA factors is mathematically inconsistent, introducing cross‑term aggregation error. In contrast, approaches that preserve heterogeneous client ranks by concatenating local adapters on the server substantially increase download cost and often require merging global LoRA updates into pretrained weights on the clients, causing reinitialization lag and unstable convergence. Other approaches further increase server‑side overhead by reconstructing dense weight updates or training auxiliary models to refine aggregation error. In this work, we propose SpecTraL, spectral transformation for layer‑wise global rank discovery, that resolves these challenges within a unified design. SpecTraL stacks local LoRA modules from clients and performs orthonormal Householder Transformation of the stacked adapters directly in the low‑rank latent space, eliminating dense reconstruction of the global update and any auxiliary refinement on the server. By leveraging the Spiked Covariance Model from Random Matrix Theory, SpecTraL analytically separates the global consensus signal from non‑IID noise, discovering optimal layer‑wise global ranks without manual hyperparameter tuning. To match local ranks in subsequent rounds, we introduce a padding‑aware initialization framework that lets clients incorporate residual LoRA dimensions without re‑merging them into the pre‑trained base model. Experiments on federated fine‑tuning of ViT‑B/16 and ViT‑L/16 over DomainNet and NICO++ demonstrate improved accuracy‑communication trade‑offs, reduced server computation, and elimination of hyperparameter search for rank selection. Our code is available at https://github.com/DASS‑Lab‑Group/SpecTraL
Authors:Xu Wang, Kaixiang Yao, Miao Pan, Xiaohe Zhou, Xuanyu Liu, Wenqi Zhang, Xuhong Zhang
Abstract:
Spatial intelligence is essential for agents to move from static semantic understanding toward interacting with the physical world. Many spatial tasks are grounded in continuous visual scenes, where locations, regions, and paths are more naturally expressed by pointing, marking, or drawing than by reporting precise coordinates or discrete textual symbols. Yet existing spatial reasoning benchmarks usually require coordinates, options, or text, creating an answer‑interface mismatch for image‑generation models. This makes it difficult to evaluate image‑generation models under the same task semantics as text‑output VLMs, despite their ability to externalize spatial judgments directly in pixel space. We propose ProVisE (Protocolized Visual Evaluation), a benchmark‑agnostic framework that elicits protocol‑constrained visual answers from image‑generation models and parses them into structured predictions compatible with original metrics. ProVisE also includes an Agentic builder that constructs and validates task‑specific protocols for new benchmarks. We further introduce SpatialGen‑Bench, a curated diagnostic benchmark of 470 samples across 14 spatial subtasks, four capability levels, and diverse answer forms. We evaluate representative text‑output VLMs and image‑generation models in a unified setting and validate Agentic protocol construction on six external spatial benchmarks. Results show that image‑generation models are competitive when spatial answers can be externalized directly in pixel space, while text‑output VLMs retain a clear advantage in compositional spatial reasoning. These findings reveal complementary strengths of pixel‑space expression and text‑based reasoning and establish a metric‑compatible testbed for studying spatial cognition in image‑generation models.
Authors:Ke Ma, Yifei Wang, Meng Wang, Tian Xia
Abstract:
Autonomous biomedical laboratories increasingly rely on visual perception to recognize, localize, and manipulate transparent plasticware, yet high‑quality real‑world datasets for this setting remain limited. The scarcity of domain‑relevant data is particularly restrictive in cluttered multi‑object scenes, where mutual occlusion and view‑dependent appearance changes remain challenging even for contemporary visual foundation models. Existing transparent‑object datasets have advanced segmentation, depth, and pose estimation, but they usually do not evaluate the combined setting of multi‑object clutter, occlusion, and calibrated multi‑view capture that characterizes real laboratory manipulation scenes. To address this gap, we present TrainsBiolab, a real‑world RGB‑D dataset of cluttered transparent biomedical objects captured as calibrated multi‑view sequences. TrainsBiolab contains 161,315 frames from 98 scenes and 1.03M instance annotations over 15 laboratory object types, including 6D poses, full and visible masks, depth, and per‑frame camera calibration. The dataset is organized along three axes that reflect operational difficulty: object category, the total number of objects in a frame, and camera viewpoint. We further define dataset‑centric benchmarks for segmentation, depth estimation and completion, and 6D pose estimation, and report a system‑level robot manipulation evaluation enabled by the released annotations and calibrations. By focusing on repeated transparent instances, clutter, and multi‑view laboratory capture, TrainsBiolab provides a resource for segmentation, depth estimation, 6D pose estimation, and multi‑view reasoning in autonomous laboratory manipulation. Project page: https://dualtransparency.github.io/TransBiolab/.
Authors:Daiqing Wu, Dongbao Yang, Jiashu Yao, Hongrui Zhang, Can Ma, Yu Zhou, Sicheng Zhao
Abstract:
Affective Image Content Analysis (AICA) aims to recognize and understand emotions elicited by visual content, representing an indispensable step toward Artificial General Intelligence (AGI). However, despite the rapid progress of Multimodal Large Language Models (MLLMs), systematic evaluation of their visual emotional intelligence remains largely absent from recent model releases. We attribute this gap to a structural mismatch between conventional AICA paradigms and the open‑ended, instruction‑driven nature of MLLMs, where further analysis reveals four major limitations: omission of plausible responses, limited emotion taxonomies, neglect of contextual factors, and labor‑intensive annotation. To overcome these barriers, we introduce Emotion Statement Judgement (ESJ), a statement‑verification formulation that preserves the expressiveness of the input space while constraining outputs to discriminative judgements. We further develop INSETS, a labor‑efficient pipeline that instantiates ESJ at scale by constructing INSETS‑462k and supporting MVEI, a rigorously refined benchmark spanning sentiment polarity, emotion interpretation, scene context, and perception subjectivity. Beyond evaluation, we build EmObserver, an emotion‑oriented MLLM optimized on ESJ through an elaborate multi‑stage recipe. Extensive evaluation of broad‑spectrum MLLMs on MVEI reveals fine‑grained insights into current artificial visual emotional intelligence, while experiments on multiple AICA benchmarks demonstrate the accuracy, generalization, and reasoning faithfulness of EmObserver. Collectively, these results establish ESJ as a practical formulation, MVEI as a comprehensive benchmark, and EmObserver as an advanced baseline for advancing MLLM‑oriented visual emotional intelligence. Code will be released at: https://github.com/wdqqdw/EmObserver.
Authors:Masaki Murooka, Ryoichi Nakajo, Keisuke Shirai, Tomohiro Motoda, Hanbit Oh, Ryo Hanai, Yukiyasu Domae
Abstract:
End‑to‑end visuomotor policies provide little opportunity for humans to understand or correct the policy's visual attention. We propose GuidedAttention, a visuomotor imitation learning framework that introduces interpretable and correctable visual attention as an explicit intermediate representation. Task‑relevant attention keypoints are predicted from camera images and condition a diffusion‑based action policy. Users can inspect and optionally correct selected keypoints once at rollout initialization, after which the corrected attention is automatically propagated throughout execution by a tracking module. Experiments in simulation and the real world demonstrate that GuidedAttention consistently improves robot manipulation performance, particularly under positional and appearance out‑of‑distribution (OOD) conditions. https://mmurooka.github.io/guided‑attention‑project‑page
Authors:Yimin Fu, Yuefeng Bai, Baicheng Pan, Zhunga Liu, Michael K. Ng
Abstract:
Adversarial attacks against large vision‑language models (LVLMs) serve as an effective means of assessing their robustness in cross‑modal semantic understanding. Existing studies mainly focus on corrupting visual inputs to induce predefined erroneous responses in general vision‑language tasks, whereas corresponding investigations in remote sensing fields remain largely underexplored. Compared with natural image understanding, remote sensing image interpretation requires joint reasoning over local discriminative cues and global scene context. This poses additional challenges to achieving transferable semantic manipulation toward specified responses under black‑box settings. To tackle these challenges, we propose GeoThreat, a transferable targeted adversarial attack method against LVLMs for remote sensing image interpretation. Specifically, GeoThreat modulates adversarial representations in accordance with the target content at both conceptual and perceptual levels. The class tokens from surrogate image encoders are employed as conceptual representations, while perceptual representations are distilled from patch tokens of the adversarial example through collaborative importance estimation. Beyond merely rolling out attention scores across layers, we incorporate adversarial‑target similarity gradients to more faithfully characterize the relevance of local visual cues to the intended semantic manipulation. The perceptual representations are then dynamically aligned with target patch tokens in a cross‑attentive manner, facilitating the adaptation of local cues toward designated semantic details. Finally, adversarial perturbations are iteratively updated via ensemble‑based joint optimization of conceptual calibration and perceptual adaptation. Extensive experiments across diverse LVLMs demonstrate the superiority of GeoThreat in both transferability and controllability.
Authors:Abhijeet Narang, Kartik Kuckreja, Shreya Ghosh, Muhammad Haris Khan, Usman Tariq, Jianfei Cai, Abhinav Dhall
Abstract:
Deepfake detection is moving beyond binary classification decisions toward systems that can also explain the visual evidence supporting those decisions. This transition is important for real‑world verification settings, where diverse users need to understand not only whether an image is manipulated, but also why it is considered suspicious. The Explainable Deepfake Detection Challenge at ACM Multimedia 2026 is designed to benchmark this joint capability. Built on XPlainVerse, a million‑scale benchmark for explainable deepfake detection, the challenge evaluates methods on image classification and grounded natural‑language explanation generation. Participants submit a real/fake label together with two explanations for each image: a detailed complex explanation for technical users and a concise simple explanation for general users. The evaluation combines classification metrics with semantic similarity, simplicity, and intent‑aware grounding metrics that assess whether explanations identify the relevant manipulated entities and supporting visual evidence. The methodologies developed through the challenge will contribute to the development of next‑generation explainable deepfake detectors. Evaluation script, baseline models, and accompanying code are available on https://github.com/Abhijeet8901/XPlainVerse‑ACMChallenge.
Authors:Zibin Lin, Shengli Zhang, Taotao Wang, Yihan Xia, Deen Ma, Guofu Liao
Abstract:
Agent Skills package reusable procedural knowledge as external artifacts for frozen language‑model agents, yet existing optimizers do not jointly resolve where a failure occurs in a workflow, which mechanism caused it, and how relevant knowledge from third‑party Skills should be reused locally. We introduce Workflow‑Localized Mechanism Learning (WML). Its Node‑‑Mechanism Attribution identifies the failed workflow node, implicated mechanisms, and smallest valid edit target, routing single‑mechanism defects to L3 resources and relational defects across mechanisms to L2 composition protocols. A six‑module Workflow‑Guided Skill Optimization (WGSO) loop then selects provenance‑ and scope‑aware third‑party knowledge, applies bounded patches, evaluates candidates, and stores verified outcomes in optimizer‑side memory. On SpreadsheetBench, WML reaches 90.33 +/‑ 1.53 and 74.67 +/‑ 3.51 Hard Accuracy with DeepSeek and Qwen3.6‑Flash, respectively; without additional optimization, the learned Skills transfer to WikiTableQuestions with 84.00 +/‑ 2.00 and 83.00 +/‑ 2.00 Denotation Accuracy. On Compiler‑Supported50, WML attains both the highest hard‑PASS rate and the lowest cost per successful task; compiled execution sharply reduces tokens and calls relative to a direct SkillAgent while retaining most of its successful tasks. Code and artifacts are available at https://github.com/xiaolin9595/workflow‑localized‑mechanism‑learning.
Authors:Nico Hezel, Kai Uwe Barthel, Bruno Schilling, Konstantin Schall, Andre Moelle, Klaus Jung
Abstract:
The annual SISAP Indexing Challenge benchmarks Approximate Nearest Neighbor Search (ANNS) algorithms under rigorous constraints. This paper presents our submissions for the 2026 edition, addressing both k‑Nearest Neighbor Graph (kNNG) construction on 1024‑dimensional BGE‑M3 embeddings (Task 1) and Maximum Inner Product Search (MIPS) on unnormalized Llama‑3.2‑8B features (Task 2). To optimize construction speed, we utilize Equi‑Voronoi Polytopes (EVP) for efficient quantization, supplemented by targeted reranking strategies to maintain high recall. For MIPS, we transform the asymmetric inner product problem into a Euclidean search space via dimensionality augmentation. To reduce query latency and optimize memory access, we introduce a 1D presorting mechanism via Fast Linear Assignment Sorting (FLAS) prior to graph construction. This significantly improves spatial locality and cache hit rates during subsequent graph traversal. Source Code: https://github.com/Visual‑Computing/sisap26‑deglib
Authors:Seolhee Lee, Minsu Kang, Yangsun Lee, Woosun Min, Choonghyeon Lee, Namhyun Cho
Abstract:
Advances in AI‑based voice conversion have enabled a wide range of media applications, including films, audiobooks, and games. However, most research and public benchmarks still focus on natural human speech, leaving designed vocalizations, such as monster growls and robotic voices, underexplored, partly due to the lack of publicly available resources. To address this gap, we introduce the Designed Vocalizations Dataset, constructed by curating diverse raw vocal sources, including speech and animal vocalizations, and applying professional vocal effects processing to produce corresponding effect modified variants. We further provide a standardized test set with explicit seen/unseen splits over source timbre groups and preset styles to assess generalization under controlled conditions. Finally, we report baseline benchmark results to support reproducible evaluation and future research. The dataset and demo samples are available at https://ncai‑official.github.io/speech/publications/designed‑vocalizations‑dataset/.
Authors:Lin Tian, Marian-Andrei Rizoiu
Abstract:
Grievance is one of the warning signs analysts look for when assessing threats of violence. It is increasingly measured at scale from online text, most often with word‑level lexicons like the Grievance Dictionary that score by matching weighted terms. Such matching is a fast and transparent proxy, but it cannot resolve whether a term is asserted, quoted, negated, or condemned. These lexicons are also often evaluated on pools enriched with the very examples they retrieve, so a high score partly reflects agreement with the lexicon's own selection rule. Examining a five‑language, 2,000‑item evaluation pool, we find its halves separated almost perfectly by the lexicon itself: every item labeled ``random'' is in fact lexicon‑negative, so the lexicon's apparent macro‑AUROC of 0.686 collapses to a 0.500 floor fixed by construction. We keep the dictionary's 22‑construct ontology but replace term matching with context‑reading models, evaluated on a non‑circular benchmark that separates unconditional‑random, lexicon‑positive, and lexicon‑negative strata across five languages. Reading the full post rather than the target sentence alone helps most where the lexicon is silent, raising average precision on lexicon‑negative text from 0.14 to 0.20, with the largest gains on quoted, implicit, and cross‑sentence grievance. Together, these results show that grievance is measured more faithfully by reading the surrounding context, and more honestly when tested on text the lexicon did not select. We release our code and benchmark at https://github.com/behavioral‑ds/multilingual_grievance.
Authors:Jiyou Shin, Youngjin Seo, Jaeseog Won, Sungwon Seo, Hyunjun Kim, Seokmin Yoon, Tuan Luong, Hyungpil Moon
Abstract:
Learning‑based manipulation policies usually predict robot actions from sensory observations and leave their execution to a separate low‑level controller. In rigid contact, this separation can be problematic: the same motion to a virtual target or compliant motion command can lead to unstable contact, tracking error, excessive loading, or tool damage, depending on the low‑level controller. In this paper, we propose a Unified Robot Control‑Policy Framework (URF), which connects compliant action prediction with unified impedance‑admittance control. Given multimodal observations, URF predicts a virtual target, a stiffness matrix, and an impedance‑admittance switch ratio. The switch ratio determines when the controller should behave more like admittance control for accurate motion tracking and when it should move toward impedance control for safer rigid contact. Because demonstration data do not provide ground‑truth environment stiffness, we construct switch‑ratio labels from measured contact forces and use them to supervise controller‑mode prediction. Across box‑flipping and line‑pressing tasks, URF achieves higher task success rates while reducing failure modes observed with admittance‑only execution, including rapid force buildup, large force oscillations, tool breakage, and robot safety stops. These results suggest that contact‑aware policies benefit from predicting not only compliant actions but also the controller behavior used to execute them. Project page: https://jiyou384.github.io/urf_project_page/
Authors:Tencent WorkBuddy Bench Team, Siqi Cai, Shaopeng Chen, Xiang Fei, Yong Mao, Zihan Xu, Zhiheng Lyu, Zhijian Shao, Yuchen Shi, Shuwen Zhang, Chaofan Qiu, Linjie Che, Xiaoxi Zhao, Feng Wu, Kai Zhang, Chaofan Zhu, Yubin Qi, Xiaoyun Liang, Peijie Dong, Yunhao Zhang, Yuanjie Zhu, Ling Jiang, Xianjun Zhang, Zhehang Chu, Anyuan Sang, Zhen Feng, Sen Nie, Shi Wu, Yuanzhen Xu, Xin Li, Ning Yang, Zhiqiang Dong, Hande Dong, Qiang Lin, Yi Liu, Yunsheng Wu, Ke Li, Xing Sun
Abstract:
We introduce Tencent WorkBuddy Bench, a multi‑domain evaluation suite for coding agents; this report documents its construction methodology, scoring protocol, and a cross‑model leaderboard. At its core is a unified evaluation framework for constructing and running distribution‑informed coding‑agent tasks across four work domains ‑ Code, Web, Office, and Security. Rather than adapting public issue text, every task is reverse‑engineered from a real commit, pull request, or business scenario and rewritten as a short, colloquial, role‑played request, so that a task's prompt is not recoverable by web‑searching the underlying issue, pull request, or commit thread. Because the dataset is released openly ‑ task directories, environment images, evaluation harness, tests, and reference solutions ‑ contamination resistance rests on this construction together with dataset versioning rather than on secrecy. The four subsets ‑ repository‑level engineering, front‑end development, office and business workflows, and red‑/blue‑team security ‑ probe complementary facets of real work, each with its own verification style. All are packaged in a uniform task‑directory format and run, under a uniform and reproducible protocol, on two agent harnesses (CodeBuddy Code and Claude Code); the full open release makes the benchmark reproducible end to end and directly auditable, since any third party can re‑run each task and inspect its content. Because each subset uses a different scoring instrument, scores are not comparable across subsets and the suite reports no suite‑wide average. We report a cross‑model leaderboard across several model families.
Authors:Katsuki Tanaka, Koichi Ito, Takafumi Aoki, Masakazu Fujio, Yosuke Kaga, Kanade Oshima, Kenta Takahashi
Abstract:
Age estimation from finger vein images has been widely considered impractical due to severe demographic biases in public datasets and physiological confounding factors like gender. To overcome these limitations, we propose MAGE‑Vein, a novel multi‑instance, multi‑task learning framework. Our approach extracts robust structural aging signs by employing a hybrid feature‑level fusion of three fingers, effectively suppressing local imaging noise. Furthermore, simultaneous optimization of gender classification conditions the network to effectively eliminate gender‑specific vascular variations. Evaluated on a demographically balanced dataset of 402 subjects, MAGE‑Vein achieves a mean absolute error of 6.12 years and a correlation of 0.880. Our results not only overturn the conventional consensus regarding the limitations of the finger vein modality but also demonstrate that previous estimation failures were primarily artifacts of biased public datasets. Our code is available at https://github.com/gsisaoki/MAGE‑Vein.
Authors:Junhao Chen, Xinghao Chen, Henghaofan Zhang, Zihao Qiao, Saining Zhang, Yongzhi Li, Ruqi Huang, Sisi Li, Yimin Sheng, Jianyi Zhu, Hao Zhao
Abstract:
Editable 3D scene creation requires object instances and lights that can be inspected, moved, and imported into standard engines, yet existing single‑image methods largely stop at room‑scale geometry, baked/global illumination, or text‑driven generation. We introduce Lumera (Light‑aware Unified Engine‑native Reconstruction and Assembly), a benchmark and reference pipeline for engine‑native, light‑aware 3D scene parsing from a single image. Lumera‑2K is built from 2,513 UE5 projects and provides 3.73M components, 63M object instances, 102.6K engine‑native parametric lights, and 95.1K camera views. On this data, Lumera‑Box and Lumera‑Light adapt VLM to parse object boxes and parametric light tuples (x,y,z,r,g,b,I), which are assembled with per‑object mesh reconstruction, HDR environment estimation, and a bounded agentic refinement loop. In a sanitized box benchmark against DetAny3D, SpatialLM, N3D‑VLM, and WildDet3D, Lumera‑Box obtains the strongest overall detection, geometry, semantic, and layout scores (merged mAP 0.1141, IoU‑B 0.2472, F‑score 0.2762), while WildDet3D remains stronger on anchor recall. For lights, Lumera‑Light recovers almost all non‑empty scenes (recall 0.998) but remains limited at individual‑light localization (F1 0.209 at 0.5 m); matched lights have median position error 0.261 m, median ΔE2000 4.59, and intensity Pearson r=0.628. These results establish parametric lights as a measurable editable‑scene target and expose remaining bottlenecks in relation structure, light recall/intensity, and cross‑engine generalization.
Authors:Hiroki Tamba
Abstract:
Position bias in multiple‑choice LLM evaluation is widely cited as a confound in capability comparisons, but published measurements rely on single answer‑order shuffles whose results confound the bias signal with content‑level noise and sampling stochasticity. I introduce inspect_permute, an open‑source extension to the inspect_ai evaluation framework that runs exhaustive answer‑order permutations per question and reports the chi‑squared / Cramer V signature of position bias with bootstrap confidence intervals. I apply the tool across four vendors (gpt‑4o‑mini, claude‑haiku‑4‑5, gemini‑2.5‑flash, grok‑3) on five MMLU subjects, 24,000 API calls under temperature‑0 generation, with falsifier predictions pre‑registered via a public SHA‑256 hash before half the data was observed. Position bias turns out to be statistically detectable only within a roughly 60‑95% base‑accuracy Goldilocks zone. Below it, processing‑load dominance swamps subject‑specific signal; above it, ceiling effects compress the variance below the chi‑squared test resolution. Detectable cells separate into two mechanism types: monotone A‑to‑D decrease (processing_load, in low‑tier models) and non‑monotone D‑drop (content_ambiguity, in a narrow capability band). Standard MMLU places every frontier‑tier model above the detection band, so absence of signal there should be read as not measurable, not unbiased. Together with the ceiling‑effect characterisation in arXiv:2606.26185, this work brackets the detectable region of position‑bias measurement and makes the field central question askable in a verifiable form. Package, data, preregistration under MIT.
Authors:Zhensheng Jin, Xin Dai, Zhenghao Liu, Chaojun Xiao, Huiyuan Xie, Yu Gu, Ge Yu, Maosong Sun
Abstract:
Large language models increasingly rely on long‑form reasoning for complex tasks, yet their reasoning traces may drift away from the supplied context when evidence is sparse, noisy, or in conflict with parametric knowledge. Existing grounding methods either attach citations after generation or encourage evidence retrieval inside the trace, but they often do not ensure that cited content is sufficient for the local inference and final answer. We propose REFACT, an adaptive fact‑restatement citation framework that trains models to decide when a reasoning step needs contextual grounding and at what granularity source facts should be restated. This design avoids both unsupported inference and indiscriminate fact copying by turning citations into answer‑supporting intermediate states. REFACT is optimized with a two‑stage SFT‑to‑RL pipeline in which a citation‑utility reward encourages cited facts to be well‑formed, source‑traceable, and answer‑sufficient. Experiments on LongBench, LV‑Eval, and ConFiQA show that REFACT improves long‑context QA and counterfactual faithfulness while substantially reducing token consumption. Further analysis shows that REFACT preserves more answer‑bearing evidence with fewer restated facts, yielding reasoning traces that are denser rather than longer. All code and data are available at https://github.com/NEUIR/REFACT.
Authors:Jaber Jaber, Osama Jaber
Abstract:
Memoir combines per‑sample fast memory, shared slow parameters, variable‑depth latent recurrence, and a future‑latent energy objective. We test its riskiest coupling: each pondering iteration may rewrite the fast tier that the same iteration reads. On procedural associative recall with key interference, we compare a coupled arm against an otherwise identical read‑only pondering arm. Both arms contain 81,738 parameters, including 76,362 trainable parameters, and use matched declared forward multiply‑accumulate counts, data, optimizer, schedule, and seeds. After 240 training steps across 12 seeds, coupled recall is 0.5203 with a 95 percent interval of [0.4522, 0.5883], while read‑only recall is 0.6557 with [0.5953, 0.7160]. The arms are paired per seed, and the read‑only lead of 0.1354 gives a paired t of 3.23 on 11 degrees of freedom with a 95 percent interval of [0.0431, 0.2277] on the difference, winning on 10 of 12 seeds. After 960 steps across 8 seeds, both arms reach 1.0000, so the measured effect is a learning‑speed penalty at a fixed budget, not a demonstrated capability penalty. That longer control is ceiling limited, leaving convergence on a non‑saturating task unmeasured. A predicted failure in which memory rewriting corrupts the energy signal did not occur: the energy margin grew and held. Kernel restructuring also reduced delta‑rule forward time from 0.907 ms to 0.351 ms on the stated device. Code and evidence are available at https://github.com/RightNow‑AI/Memoir
Authors:Michael Xu, Jorge Leandro, Sudha Rao, Weijia Xu, Nebojsa Jojic, Gabriel DesGarennes, Chris Quirk, Bill Dolan
Abstract:
We introduce Rushes, a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments. Rushes is collected through a game interface where users interact with AI‑generated branching narratives and select one choice from a small, explicit candidate set at each decision point. Each interaction logs the full candidate set, the user's choice, and the evolving narrative context, yielding time‑ordered trajectories with persistent user‑level identifiers. Rushes contains 44,226 decision events from 8,167 unique users across six games, capturing sequential, personalized engagement behavior rather than static judgments. We show that user choices exhibit structured, non‑random patterns, quantified by a low choice entropy relative to a uniform baseline. We position Rushes as a diagnostic benchmark for pluralistic alignment and demonstrate a robust Engagement Gap: state‑of‑the‑art LLMs, including GPT‑5, fail to outperform simple baselines. While classical Matrix Factorization (SVD) captures measurable personalized signal (37.7%), frontier LLMs (34.23%) struggle to even match the Popularity Baseline (36.4%) on event‑level choice prediction. This gap suggests that single, population‑level objectives, like those used in modern RLHF, appear insufficient to capture heterogeneous, context‑dependent engagement signals. As a result, even highly capable models default to majority preferences rather than adapting to individual trajectories. We release Rushes to support research into pluralistic alignment and sequential decision‑making in generative systems. The full code for the platform and dataset will be available here: https://github.com/microsoft/rushes
Authors:Pavel Golikov, Evgenii Opryshko, Gennady Pekhimenko, Mark C. Jeffrey
Abstract:
We introduce ARBIGRAPH, a benchmark generator for evaluating whether tool‑assisted language agents can retain, update, compose, and discard task‑relevant context across extended reasoning workflows. ARBIGRAPH represents each task as a natural‑language problem with an executable Python solver, and composes tasks through typed intermediate states, instantiated here as scalar and list values. This design enables controllable task graphs whose length, dependency structure, distractor count, and value type can be varied while preserving exact automatic verification. We instantiate ARBIGRAPH with math, GSM‑style word‑problems, and Python‑tracing task categories, and evaluate a Qwen3.5‑27B tool‑assisted agent across four topologies. The results show high accuracy on isolated tasks but substantial degradation on more complex dependent tasks: accuracy drops by up to 33.3% on branching chains of dependent math tasks. This shows that ARBIGRAPH exposes failures that are not visible from single‑task evaluation alone. Our code, generated datasets, and evaluation results are available at https://github.com/pavelgolikov/ArbiGraph.git
Authors:Miguel P. Bento, João F. Seabra
Abstract:
Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization. GaugeQuant leverages this in‑training by introducing a LogSumExp term to the loss that breaks the symmetries, thus selecting a basis that minimizes activation outliers. A stop‑gradient operator ensures that only rotation matrices are updated, yielding the language modeling objective completely unaltered. Our requires no specific calibration data, no quantization simulation, and adds negligible training overhead. With the LLaMA‑2 7B model under W4A4 quantization with group size 128, perplexity drops from 8.22 to 6.73, competing with post‑training methods that require frozen models and calibration datasets. Under W4A16, perplexity drops from 11.16 to 5.45. Code is available at https://github.com/MPedraBento/gauge‑quant.
Authors:Zinan Li, Yiyang Ling, Yuming Gu, Binghao Huang, Chenhao Liang, Sharfin Islam, Hisham Bedri, John Chirikjian, Yunzhu Li, Stefanos Nikolaidis, Daniel Seita
Abstract:
The sense of touch is central to manipulation, especially when vision is occluded or ambiguous. Although combining vision and touch improves manipulation, learning robust visuo‑tactile policies requires substantial tactile data. Such data remains scarcer than visual data, because tactile sensors are fragile, specialized, and hard to standardize. To address this, we present Feature‑Extracted Latent Tactile (FELT), a learning‑based framework that synthesizes per‑finger pressure tactile images from RGB observations, reducing the need for tactile‑equipped data collection. FELT uses a large frozen visual encoder and a lightweight query decoder to predict tactile signals in a single feed‑forward pass. To respect the physical topology of dual‑finger tactile sensors, FELT decodes the left and right tactile sensor panels through separate branches, capturing the asymmetric contact patterns during interactions such as wiping, insertion, and in‑hand rotation. At inference time, FELT only requires RGB data, allowing us to augment existing vision‑only data with tactile observations, either as generated tactile images or as latent tactile features. Experiments on four contact‑rich manipulation tasks demonstrate that both generated tactile images and latent tactile features improve policy success over vision‑only baselines, with latent feature requiring no real tactile sensor during policy training or deployment. Supplementary material is available on our anonymous website: https://felt‑tactile.github.io/.
Authors:Renbiao Jin, Mingxin Yang, Yutian Chen, Junhao Zhuang, Xin Cai, Mulin Yu, Linning Xu, Wenxian Yu, Danping Zou, Shi Guo, Tianfan Xue
Abstract:
Real‑world video deblurring remains challenging due to diverse motion patterns, complex degradations, and the scarcity of realistic training data, yet robust restoration is critical for downstream pipelines such as mobile imaging and 3D reconstruction. This work presents RealVDeblur, an efficient generative framework designed to improve in‑the‑wild robustness under diverse real capture conditions. First, a large‑scale, physically grounded blur synthesis pipeline is constructed from scene‑level 3D Gaussian Splatting (3DGS) assets and high‑frame‑rate videos, providing realistic training data covering both camera‑induced and object‑motion blur. Second, a video diffusion prior is leveraged for restoration; to better accommodate frame‑dependent blur variations, temporal compression in the VAE is disabled and a frame‑wise encoding scheme is adopted. For practical deployment on long videos, multi‑step diffusion sampling is distilled into an efficient one‑step generator, and a training‑free Temporal Window Mask stabilizes inference beyond the training horizon with constant memory usage. Extensive experiments on diverse real‑world benchmarks demonstrate strong perceptual quality, semantic fidelity, and temporal consistency on unseen videos, as well as improved robustness in downstream 3D reconstruction under severe motion blur. Project page: https://rbjin.github.io/RealVDeblur
Authors:Jan Linnenbrink, Jakub Nowosad, Marvin Ludwig, Anna Frederike Jablotschkin, Fabian Schumacher, Teja Kattenborn, Hanna Meyer
Abstract:
Spatio‑temporal machine‑learning modelling is an important tool in environmental research. However, machine‑learning models are highly sensitive to both the characteristics of the training data, such as its distribution, and methodological choices, including the cross‑validation strategy. Each decision has impact and implications on the model itself as well as the estimation of the model quality and applicability for certain purposes. Taking into account the large role of machine‑learning based maps of the environment in science and their transfer into practice, transparent reporting of spatio‑temporal models, ideally using standardized model protocols, is essential to enable trust, transparency and comparability. However, such protocols are currently lacking for spatio‑temporal modelling. We propose STeMP (Spatio‑Temporal Modelling Protocol) to fill this gap by serving two purposes: standardized reporting to understand the model functioning as well as providing guidance during the modelling process by pointing at critical decisions and parameters. The protocol is structured in three sections: Overview, Model and Prediction. The Overview section contains metadata, while the Model and Prediction sections go into detail, describing predictors, evaluation and software, and further relevant elements of the modelling workflow. The protocol definition is hosted on GitHub and accompanied by an R‑package (https://github.com/LOEK‑RS/STeMP). The R‑package contains a web application that can be used to fill the protocol either manually or in a semi‑automated way from provided modelling objects. Warnings are returned from the protocol when common pitfalls are encountered, which may help authors as a guide through the modelling process but also support reviewers in the assessment of modelling studies. Via GitHub, incorporation of contributions and feedback from the community is encouraged.
Authors:Liangqin Ren, Zeyan Liu, Ye Wang, Yuxin Chen, Fengjun Li, Bo Luo
Abstract:
Deepfakes, especially face‑swapping attacks, pose significant challenges to authenticity, security, and ethics across science, engineering, and society. While most existing detection/tracing approaches operate post hoc, proactive defenses that aim to intervene before deepfake generation remain limited in terms of real‑world effectiveness. In this paper, we present PhantomSeal, the first proactive defense to simultaneously protect both the identity and the context of users' images from being used in face‑swapping attacks, while supporting forensic tracing. We present a novel cloaking technique that embeds a selected identity as a stealthy identifier. This mechanism steers the deepfake generation process toward producing content that resembles the chosen cloak identity, thereby preventing successful face‑swapping while enabling effective feature‑based forensic analysis. The effectiveness and robustness of PhantomSeal is demonstrated in extensive experiments across different face‑swapping architectures and models. For example, it reduces the attack success rate of SimSwap, an advanced deepfake model, to 0.30%, and correctly identifies 97.97% of manipulated content. Codes can be found at https://github.com/LiangqinRen/PhantomSeal
Authors:Hesen Chen, Xinyu Su, Xiaomeng Yang, Yuetan Lin, Zixiong Yang, Junyi An, Fenglei Cao, Yifeng Jiao, Yunqi Zhang, Yuan Cheng, Zhiyu Tan, Hao Li, Libo Wu, Yuan Qi
Abstract:
Scientific discovery is increasingly shifting from isolated disciplines to multi‑domain reasoning, and AI for science faces a similar transition. Existing systems are either specialised for individual domains or unify scientific data mainly through text tokenisation and prompt‑based interfaces, limiting their ability to handle diverse scientific inputs, produce modality‑native outputs, and support joint understanding, reasoning, and generation across scientific domains. We introduce MKB, a unified scientific multimodal model for both understanding and generation, built around a shared Transformer backbone and modality‑tailored encoders, adapters, and decoders. MKB covers six scientific branches, including DNA, RNA, proteins, small molecules, earth science, and medical images, and supports native outputs such as biological sequences, molecular strings, meteorological fields, and segmentation masks. Training follows a two‑stage modality‑then‑language curriculum: Stage 1 aligns modality‑specific components with the frozen backbone, and Stage 2 consolidates them with the language backbone using mixed scientific and general corpora. Experiments show that MKB achieves competitive scientific understanding across biological and molecular benchmarks, produces high‑fidelity native outputs for weather forecasting, biological generation, and medical‑image segmentation, and largely retains the general capabilities of its Qwen3‑VL backbone. These results demonstrate the feasibility of the proposed paradigm, suggesting that shared‑backbone models with modality‑tailored components can provide a promising foundation for future cross‑domain scientific multimodal exploration. The model and code are publicly available at https://github.com/Shanghai‑Academy‑of‑AI‑For‑Science/MKB and https://huggingface.co/sais‑org/MKB.
Authors:Yubo Wang, Qiuyu Zhao, Zenghui Sun, Shichao Dong, Jinsong Lan, Xiaoyong Zhu, Haoyang Li, Bo Zheng, Lei Chen
Abstract:
Memory Manager models are pivotal in agent systems. Existing methods rely predominantly on LLM‑judged synthetic question‑answer (QA) pairs, making memory valuation dependent on sampled queries and the downstream reader. To address this limitation, we propose CMI‑Mem, a reinforcement learning(RL)‑based lightweight memory manager model with a hybrid reward that combines downstream QA correctness and intrinsic Conditional Mutual Information (CMI). CMI evaluates the information contributed by new conversational inputs relative to the current memory state without conditioning on a sampled QA query, thereby complementing rather than replacing QA grounding. Our codes are available at: https://github.com/Wyb0627/CMIMem , and the CMI‑Mem‑4B model checkpoint is available at: https://www.modelscope.cn/models/wyb0627/CMIMem‑4B
Authors:Chitraansh Pandey
Abstract:
Grokking ‑‑ the delayed generalization of neural networks long after they have memorized their training data ‑‑ wastes thousands of training epochs and is notoriously unpredictable. Building on the recent result that Transformer attention is formally isomorphic to a thermodynamic system, we treat the variance of attention logits as a specific heat Cv and show that its peak reliably precedes the generalization transition. We introduce CvAdamW, a drop‑in AdamW variant that monitors Cv online and injects thermal energy by dynamically scaling weight decay when a phase transition is detected. Through a strictly iterative development process we identify three failure modes ‑‑ initialization noise, mini‑batch micro‑ripples, and slingshot blinding ‑‑ and resolve them with a memorization gate and an exponential‑moving‑average shock absorber. On modular arithmetic (a+b mod 97), CvAdamW enables grokking at epoch 2802 in a 4000‑epoch budget where the baseline never groks. We further propose a scale‑invariant z‑score reformulation that removes task‑specific hyperparameters, and evaluate it across 10 paired seeds. A paired analysis shows the cold‑start variant reduces mean grokking latency by 257 epochs (6.0%; median 166 epochs; Wilcoxon p=0.049, Cohen's d=0.68, bootstrap 95% CI [53,489]), improving 8 of 10 seeds; on this single task Cv peaks before grokking in all 10 seeds. Our results indicate that neural networks may expose detectable precursors of impending generalization transitions, and that a physically motivated, proportional intervention can facilitate generalization within a fixed compute budget. Code and data are public.
Authors:Mikail Khona, Aditya Vavre, Boxiang Wang, Deyu Fu, Hao Wu, Mike Chrzanowski, Bryan Catanzaro, Dheevatsa Mudigere, Jeff Pool, Michael Lightstone, Mohammad Shoeybi, Mostofa Patwary, Nima Tajbakhsh, Tijmen Blankevoort
Abstract:
Higher‑order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computational cost and numerical stability challenges have limited adoption at scale. In this work, we adapt and enhance preconditioned gradient methods to overcome the practical challenges of large‑scale LLM pretraining. We first identify instabilities in SOAP at large batch sizes and propose algorithmic modifications including per‑step QR orthogonalization and improved preconditioning strategies that eliminate loss spikes and enable stable training in these regimes. We then present a unified empirical study of SOAP, Muon, and AdamW using update‑RMS matching to ensure fair learning rate transfer across optimizers. As part of this analysis, we empirically evaluate the orthogonalization quality of Muon. Our experiments on multi‑billion‑parameter models trained on trillions of tokens reveal that SOAP and Muon consistently outperform AdamW at the scales we tested. Notably, at batch sizes of up to 100M tokens for next‑token prediction, these optimizers maintain training stability and quality while AdamW degrades. To enable efficient training at large scale, we introduce a layer‑wise distributed optimizer compatible with Megatron‑LM. Our implementation balances memory and hides communication while avoiding approximations to the optimizer computations, thus retaining their convergence benefits. Additionally, we identify and build specific system‑level improvements to further accelerate our layer‑wise implementation. To support the research community, we release a codebase that contains emerging algorithms for optimization: https://github.com/NVIDIA‑NeMo/Emerging‑Optimizers
Authors:Jingyi Huang, Ruohan Zong, Yujun Feng, Liran Ma, Lanyu Shang, Yang Zhang
Abstract:
Reinforcement Learning from Human Feedback (RLHF) is critical for aligning Large Language Models (LLMs) with human preferences. However, its efficacy is often compromised by the inherent inconsistency and subjectivity of human annotations. Existing preference optimization frameworks, such as Direct Preference Optimization (DPO), typically treat ambiguous pairs with high annotator disagreement identically to those with unanimous consensus, forcing models to overfit to inconsistent supervision signals and leading to suboptimal alignment. In this work, we propose Reliability‑Guided Preference Optimization (RGPO), a robust framework designed to mitigate the impact of inconsistent human feedback. RGPO estimates annotator reliability and infers latent ground truth labels from noisy human feedback to identify robust preferences. Furthermore, we introduce a reliability‑aware consistency optimization that dynamically modulates the training objective based on the consensus level of annotations, ensuring the model prioritizes high‑consensus supervision signals. Extensive experiments on LLM alignment benchmarks demonstrate that RGPO effectively reduces inconsistency and noise in training data and achieves superior performance compared to widely adopted RLHF baselines. Our code and configurations are available at https://github.com/GenieHuang/RGPO.
Authors:Yufeng Wang
Abstract:
The Muon optimizer reaches the grokking threshold on modular arithmetic faster than AdamW. Prior work attributes this to "spectral‑norm constraints plus orthogonalized momentum" but does not isolate which mechanism matters. To better understand Moun's behavior, we run multi‑seed and multi‑learning‑rate sweeps to decompose and stress‑test the effect. First, an ablation shows the speedup comes from orthogonalization (the Newton‑Schulz iteration): orthogonalize‑only matches full Muon, whereas spectral‑only is no faster than AdamW and is unreliable, and this verdict holds across learning rates. Second, a mechanistic analysis finds that orthogonalizing optimizers reach generalization at roughly 3x lower spectral norm and, controlling for how much the embedding actually moves, settle into a lower‑norm solution rather than simply perturbing the embedding less. Third, reducing the Newton‑Schulz iteration count from five to one accelerates reaching the threshold but makes the grokked solution fragile, prone to transient collapse, with fragility that grows with learning rate; a single iteration is fast and stable only at small learning rate, while the canonical five iterations are the learning‑rate‑robust choice. We also show spectral scaling can be dropped at no measured cost. A methodological thread runs throughout: under a stability‑aware metric, "faster" claims about grokking optimizers can invert, so we report both first‑crossing and sustained‑grok times. To support reproducibility, we release our full training and analysis code at https://github.com/louiswang524/muon‑grokking‑frontier
Authors:Fanjin Zhang, Zhengyang Wang, Ruixuan Huang, Kefan Zhang, Amy Xin, Yuanchun Wang, Shu Zhao, Evgeny Kharlamov, Jie Tang, Juanzi Li
Abstract:
Large language models (LLMs) augmented with tools are emerging as autonomous agents capable of using Web engine, APIs, and code to solve complex, long‑horizon tasks. Current tool‑using benchmarks for information seeking on academic graphs rely on synthetic templates, simplified solution spaces, or narrow tasks such as paper‑centric tasks, leaving key challenges underexplored ‑ realistic user intent, complex multi‑step API planning, rich parameter filling for APIs, grounded answers with references, and comprehensive evaluation of both the process and the outcome. We introduce AISE‑Bench, a real‑world, full‑cycle annotated benchmark for information seeking on academic knowledge graphs. AISE‑Bench release contains 1,133 QA pairs, including query taxonomies, full API execution trajectories, validated parameters, and source‑grounded answers with reference links. To support high‑quality annotation, we design a customized agent workflow to enable annotators to plan, execute, and revise complex API workflows efficiently. We develop a comprehensive evaluation protocol measuring answer quality, reference grounding, API‑planning correctness, and execution success. Among the 14 evaluated methods, even the strongest model (PLAY2PROMPT with Gemini‑3‑Pro) achieves only moderate performance and often struggles with API planning and execution. AISE‑Bench establishes a challenging new testbed for quantitatively evaluating and improving the stepwise correctness, grounded summarization, and traceable reasoning of multi‑step API‑using LLM agents. Our code and data are available at https://aise‑bench.github.io/.
Authors:Raffi Khatchadourian
Abstract:
A financial AI agent can repeat a decision while changing the tools, order, or recorded arguments and results used to reach it. Outcome‑only evaluation misses this variation, even when it matters for replay and change control. DFAH‑Bench operationalizes the Determinism‑Faithfulness Assurance Harness (DFAH), where faithfulness means fidelity of observable execution under replay, not answer correctness. The protocol qualifies comparable, sufficiently observed replays and measures decision agreement (DAR) and tool‑path agreement (TAR) over the same eligible groups. We analyze 4,157 retrospective episodes from configurations with observed tool use across 719 synthetic compliance and financial DataOps groups, together with an argument‑aware prospective extension comprising 570 eligible episodes across 190 groups. In that extension, decisions agree 94.2‑95.1% while exact tool‑name paths agree 66.9‑69.4%, producing 25.8‑27.3 percentage‑point gaps; argument‑and‑result trajectory agreement falls to 45.0‑51.5%. Even among unanimous‑decision groups, paths vary in 66.7‑68.9% under task weighting. DFAH‑Bench makes the execution behind a stable decision visible for replay, investigation, and change review.
Authors:Jiawei Zhou, Jianwei Wang, Chenyu Zhou, Chaojian Shi, Ming Dong, Kai Wang
Abstract:
Text‑to‑SQL has advanced rapidly with large language models, but complex database queries still require reasoning beyond one‑shot generation, including multi‑step decomposition, execution‑based diagnosis, and targeted correction. We present EvoSQL, a co‑evolution framework that formulates SQL synthesis as an iterative interaction between a generator and a critic. EvoSQL maintains a contextualized candidate memory, verifies SQL candidates with both execution signals and LLM‑based critique, and updates its memory through utility‑guided aggregation. To strengthen the underlying generator‑critic pair, we further introduce a Self‑Distillation Policy Optimization (SDPO) fine‑tuning stage that injects execution‑aware supervision into modern coding LLM backbones. Experiments on Spider and BIRD show that EvoSQL consistently improves open‑source models over Maj@16 baselines, with particularly large gains on BIRD‑Dev, ranging from +1.37% for Qwen3‑4B to +9.19% for Qwen2.5‑Coder‑3B. SDPO initialization further improves selected backbones on Spider‑Test and BIRD‑Dev. These results suggest that memory‑grounded co‑evolution is an effective path toward more reliable and generalizable Text‑to‑SQL systems. Code is available at https://github.com/valleysprings/EvoSQL.
Authors:Bronislav Sidik, Chaya Levi, Nizzan Kimhi
Abstract:
Multi‑agent LLM frameworks typically fix their team topology at boot time. When an individual agent becomes overloaded at runtime, for example by mixing too many action categories, accumulating tool errors, or queueing behind too many calls, the system has no mechanism to restructure itself. We introduce Autonomous Topology Mutation (ATM), a runtime team‑mutation mechanism for multi‑agent LLM frameworks. ATM combines telemetry‑driven overload detection with three safety invariants that gate each structural change: capability monotonicity, state‑routing completeness, and shadow‑before‑live validation. ATM monitors a six‑signal Bottleneck Index that includes queue depth, context thrash, tool‑error rate, role entropy, retry‑loop rate, and cross‑agent wait time. When a warmup‑calibrated threshold is breached for multiple consecutive ticks, ATM factorises the overloaded agent into specialised sub‑agents and hot‑swaps the parent into a coordinator role while preserving its external identity. State transfer is controlled by privacy‑level‑aware routing: each memory atom is routed only to a permitted child set, or explicitly dropped with a logged reason. No candidate topology receives live traffic until it has passed a shadow validation window. On 720 DeepSeek‑V3‑driven task runs with deterministic tool stubs across four ablation conditions and three workloads, the ATM factoriser split lifts code‑task success from 3.3% to 61.7%. The full rail‑and‑distillation system reduces detected high‑privacy memory exposure under a regex classifier from 2.0 to 0.0 events per task while preserving task quality. The runtime rails carrying ATM's invariants add less than 500 microseconds of p99 latency on the agent hot path. A small live‑tool probe with real Python execution is included as an external‑validity check. The implementation, benchmark harness, and traces are open‑sourced.
Authors:Yiheng Tao, Kaiwen Cheng, Yao Lu, Chang Liu, Jie Chen
Abstract:
Large Language Models (LLMs) are fundamentally limited by representation collapse, a bottleneck that severely degrades long‑context performance. We identify that existing approaches risk drifting into one of two pathological extremes: homogenization collapse (e.g., attention sinks causing rank deficiency) and isolation collapse (e.g., local attention causing context disconnection). Through spectral analysis of attention dynamics, we derive an intrinsic trade‑off between mixing efficiency (spectral gap) and information capacity (effective rank) that standard mechanisms struggle to balance. To resolve this dilemma, we propose the Topologically Regularized Side‑Path (TRSP), a non‑invasive architectural intervention that achieves spectral balance. TRSP employs a parameter‑free Triangular Box mechanism, scaled by a lightweight, length‑aware gate, to regularize the token interaction topology. By integrating proximal coupling to preserve effective rank and distal propagation to support non‑degenerate mixing, TRSP promotes a geometrically healthier transition operator without altering core attention. Experiments show significant improvements across general capabilities and long‑context benchmarks. Notably, on NoLiMa at 8× the training length, TRSP retains 83% accuracy and surpasses the Differential Transformer and Gated Attention by approximately 30 and 50 percentage points, respectively. Code available at: https://github.com/Eziotao‑tyd/TRSP.
Authors:Faizan Iqbal
Abstract:
We present an empirical benchmark evaluating how five large language models assess multisensor physical hazard data. Testing 60 scenarios across three categories ‑ multi‑sensor joint assessment, response proportionality, and pattern disambiguation ‑ with 1,800 API calls at temperature 0.0, we find that all tested models consistently produced no precautionary warning signal across the tested scenarios where multiple sensors are simultaneously elevated below their individual safety limits, while achieving near‑perfect accuracy on single‑sensor threshold violations. All five models (ChatGPT‑4o, Gemini 2.5 Flash, DeepSeek, Kimi, Llama 3.1 8B) score near zero on Category A multi‑sensor scenarios (Q2: 0.000‑0.208; Q3: 0.000‑0.592) compared to strong performance on single‑sensor scenarios (Category B Q1: 0.975‑1.000). Structured tabular formatting shows no consistent advantage over plain prose; ChatGPT‑4o performs significantly better under prose (p = 0.001). These findings have direct implications for practitioners deploying the tested models in physical safety monitoring systems.
Authors:Jiacheng Wang, Weiyan Zhang, Guangya Yu
Abstract:
Enhancing the task‑specific capabilities of Large Language Models (LLMs) primarily requires substantial instruction‑tuning datasets. However, the sheer volume of such data imposes a considerable annotation cost, and a lack of optimization methods for tailoring LLMs to specific tasks. To address the above issues, we propose a Planning framework for constructing Extractive‑based LLMs called PlanE, which includes data decomposition, instruction tuning, and prompt inference. Additionally, we introduce a Data‑Tuning‑Inference (DTI) planner, aimed at selecting the optimal base‑LLM and its DTI combinations for specific datasets to improve construction efficiency. The experimental results demonstrate the effectiveness of our PlanE from two views: (1) across different datasets using the same base‑LLM, and (2) on the same dataset using different base‑LLMs. Furthermore, we validate the generalizability of the proposed DTI planner under different optimization objectives. The codes are publicly available at https://github.com/gugugu‑469/PlanE.
Authors:Zishan Shao, Lixun Zhang, Kangning Cui, Yixiao Wang, Ting Jiang, Hancheng Ye, Qinsi Wang, Zhixu Du, Yuzhe Fu, Fan Yang, Danyang Zhuo, Yiran Chen, Hai Helen Li
Abstract:
Large language models (LLMs) handle many tasks with one set of parameters, but under KV‑cached inference it is unclear what task‑general structure, if any, is used at decode time rather than during prefill. We propose DecodeShare, a protocol that identifies a low‑dimensional subspace consistently shared across tasks in decode‑time hidden states, and then tests its causal role by removing that subspace only during decoding. In our experiments, disturbing the discovered shared subspace degrades decision performance far more than disturbing either a prefill‑derived or random subspace under the same intervention budget. We further show this decode‑shared subspace has practical consequences for activation steering: common steering directions can overlap the task‑general decode channel. Projecting out this shared subspace directly separates the functional roles of the two components, while evaluating steering vectors at decode‑time yields more reliable signal for downstream deployment than prefill‑based proxies. Despite its compactness, the shared subspace can serve as a high‑leverage causal channel at decode time. Code is available at: https://github.com/Zishan‑Shao/decodeshare.git.
Authors:Yanhua Jiao, Tianyi Wu, Xiaoxi Sun, Yulin Li, HuiLing Zhen, Libo Qin, Baotian Hu, Zhuotao Tian, Min Zhang
Abstract:
While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds. These thresholds, necessitated by the Joint Probability Dependence Error (JPDE), result in redundant denoising iterations and suboptimal inference speeds. To overcome this, we propose DC‑Leap, a training‑free framework that enables reliable acceleration of dLLMs in the moderate‑confidence regime. DC‑Leap introduces a Dynamic Contiguous Verification strategy that integrates strictly‑ordered causal constraints into the parallel decoding process. By progressively validating token dependencies, this mechanism effectively neutralizes the JPDE, enabling reliable acceleration with comparable performance. Furthermore, DC‑Leap incorporates the draft‑guided decoding mechanism, where the draft helps extend the context by leaping forward across multiple tokens, providing look‑ahead context and retaining the structural benefits of bidirectional attention during inference. Extensive experiments on standard benchmarks demonstrate that DC‑Leap achieves substantial speedups, up to 53.19x on MBPP for long‑sequence generation, and up to 105.02x when combined with KV‑Cache with comparable generation quality. Code is available at https://github.com/ffh‑wyls/DC‑Leap .
Authors:Anmol Guragain, Marcos Estecha-Garitagoitia, Luis Fernando D'Haro Enríquez, Ricardo de Córdoba
Abstract:
This paper describes our system for the EEUCA 2026 Shared Task on toxicity classification in gaming chat. We implement a three‑stage pipeline combining an ensemble of two compact transformers (DeBERTa‑v3‑base, 184M; XLM‑RoBERTa‑base, 278M) with a Linguistically‑Informed Mediator (LIM) that resolves inter‑model disagreements through corpus‑backed lexical normalization, class‑conditional unigram scoring, multilingual profanity detection, and agentive targeting analysis grounded in speech act theory. The LIM specifically targets the minority classes (Hate \& Harassment, Threats, and Extremism), which are the most safety‑critical categories in real‑world gaming moderation. To address the extreme class imbalance (1,450:1 Non‑toxic to Extremism ratio), we introduce a two‑stage data augmentation strategy using only the provided training data. Our system achieves a Macro F1 of 0.6441 and accuracy of 0.9062 on the official test set, ranking 3rd in Macro F1 and 1st in accuracy among all teams. The proposed pipeline is domain‑portable: adapting to other gaming platforms requires substituting only the game‑specific entity lexicon. Code is publicly available at https://github.com/Anmol2059/thaulab\_EEUCA.
Authors:Sebastien Kawada
Abstract:
Political evasion is difficult to detect because evasive answers often appear cooperative while avoiding concrete commitment. We present AsymVerify, a confidence‑gated verification system for SemEval‑2026 Task 6, a three‑way classification of Clear Reply, Ambivalent, and Clear Non‑Reply responses. AsymVerify scored 0.85 Macro F1 on the evaluation split (D_eval, n=237), placing 2nd out of 41 teams on the official leaderboard. The system first classifies each question‑answer pair, then selectively applies downgrade verification (CR/CNR ‑> AMB) or upgrade verification (AMB ‑> CR) to low‑confidence predictions. Development analysis shows that errors concentrate at the Ambivalent boundary in both directions, motivating this asymmetric two‑verifier design while confidence gating keeps additional inference cost low. On D_dev (n=308), AsymVerify with GLM‑4.7 gains +17.1 Macro F1 over single‑pass classification at 1.48 calls/example, and the upgrade verifier alone improves every tested LLM backend on D_dev by +6.8 to +15.2 Macro F1 over its single‑pass baseline. Code is available at https://github.com/kaons‑research/AsymVerify‑ACL.
Authors:Phuong Huu Vu Tran, Long Minh Vo, Son Nguyen Minh Le, Hoang Van
Abstract:
We present LLM‑INSTRUCT, the winning system for the UZH Shared Task at ArgMining 2026 on paragraph‑level argument mining in UN and UNESCO resolutions. The task requires paragraph‑type classification, prediction of a subset of 141 official tags, and directed relation prediction under a strict JSON schema setting using only open‑weight models up to 8B parameters. We frame the task as constrained structured prediction. The system first narrows the candidate tag space with metadata‑aware dense retrieval, then applies constrained decoding with per‑dimension caps, escalates only uncertain cases to a three‑agent debate branch, and finally validates the output schema. On the official leaderboard, LLM‑INSTRUCT ranked 1st overall, with 1st in F1 and 5th in LLM‑as‑a‑Judge. During development, our configuration search further improved Task 1b Micro‑F1 from 35.83% to 40.08% while keeping the internal Task 2 score at 4.421. The main lesson is simple: reducing the decision space before generation improves both accuracy and submission robustness. Our code and supporting scripts are publicly available at: https://github.com/LLM‑Instruct‑at‑UZH‑Shared‑Task‑2026/Method
Authors:In Cho, Jeonghwan Cho, Mijin Yoo, Gim Hee Lee, Seon Joo Kim
Abstract:
3D Gaussian Splatting (3DGS) achieves high‑quality novel‑view synthesis by optimizing freely placed primitives in 3D and adaptively densifying them in under‑reconstructed regions. However, this scene‑adaptive capacity allocation is largely lost in existing feed‑forward 3DGS methods, which commonly regress Gaussians at input pixels and lift them along camera rays. Such pixel‑aligned formulations make the number and placement of primitives depend on image resolution and input viewpoints rather than scene complexity, resulting in dense and often redundant Gaussian sets. We present ATSplat, a feed‑forward 3DGS framework that restores the adaptive allocation capability of 3DGS optimization through Adaptive 3D Tokens. ATSplat first lifts coarse patch‑level depth and camera cues into sparse 3D anchor tokens, forming a compact scaffold of the scene. Each token is then regressed into local Gaussians with learnable 3D offsets, decoupling primitive placement from input image grids. An Adaptive Token Expansion module predicts a token‑level uncertainty score, supervised by rendering error maps, and selectively expands high‑uncertainty tokens through learnable expansion layers. This sparse‑to‑adaptive formulation enables ATSplat to concentrate primitives in challenging regions while maintaining a compact representation. Experiments on two representative datasets, RealEstate10K and DL3DV, show that ATSplat achieves state‑of‑the‑art rendering quality while reducing the number of Gaussians by more than 5.7× compared with dense feed‑forward 3DGS methods. From 12 input images at 512 × 960 resolution, ATSplat completes reconstruction in less than a second using a single commercial GPU, and renders high‑quality novel views at 1136 FPS (512 × 960) with only 311K Gaussians.
Authors:Nethmi Muthugala, Supryadi, Surangika Ranathunga, Nisansa de Silva, Ruijie Tao, Ovindu Gunatunga, Pengyun Zhu, Shaowei Zhang, Jingting Zheng, Deyi Xiong
Abstract:
Value alignment of Large Language Models (LLMs) has been shown to be culturally biased toward Western norms. This results in the mishandling of local values in multilingual societies such as Sri Lanka that have their unique cultural dynamics. Existing benchmarks overlook Sri Lankan‑contextualized values in its official language Sinhala, hindering culturally sensitive evaluation and fine‑tuning. To bridge this gap, we propose LKValues, the first survey‑grounded resource suite for Sri Lankan value alignment. From a trilingual survey of 205 respondents, blending adapted global frameworks and LLM‑elicited local constructs, we derive 40 majority‑endorsed societal values. Using these values, we construct LKvaluesIT, a Sinhala‑English news‑derived instruction corpus containing 150k scenario‑based instances, and LKvaluesBench, a value‑sensitive evaluation benchmark of 1,000 instances. We evaluate a set of proprietary and open‑weight LLMs with LKvaluesBench. We fine‑tune three open‑weight base models (Qwen3.5‑4B‑Base, Qwen3.5‑9B‑Base, and Aya‑Expanse‑8B‑Base). Our experiments show that newer and larger LLMs still exhibit low‑resource and cultural value‑alignment gaps. LKValues fine‑tuning improves Qwen‑family models in English and Sinhala, reducing invalid outputs and cross‑lingual disparities, though gains remain model‑family dependent. These highlight LKValues efficacy in embedding Sri Lankan values, offering a replicable pipeline for low‑resource, country‑specific pluralist value alignment. The dataset is publicly available at https://github.com/NextME14/LKValues.
Authors:Junhao Zhuang, Shiyi Zhang, Yuxuan Bian, Yaowei Li, Yawen Luo, Yijun Liu, Weiyang Jin, Songchun Zhang, Xianglong He, Xuying Zhang, Haoran Li, Haoyang Huang, Zeyue Xue, Nan Duan
Abstract:
Recent autoregressive video diffusion methods are increasingly built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground‑truth video contexts. This reduces exposure bias, but the historical key‑value cache is still used by future frames only as frozen rollout state. As a result, future losses cannot supervise how earlier generated latents should be written into more useful keys and values for later video‑latent generation. We call this the historical context‑gradient gap. We propose Self Gradient Forcing (SGF), a two‑pass training strategy that restores this missing supervision signal without backpropagating through the full serial rollout. Pass 1 performs a no‑gradient autoregressive rollout matching inference and, at a sampled denoising exit step, records both the self‑generated context and the noisy latents fed to the model. Pass 2 performs parallel context‑gradient reconstruction for the recorded exit step. The generated context is used as stop‑gradient clean‑latent input, while the model recomputes the context KV representations and future‑to‑context causal attention. Thus, SGF provides the missing memory‑writing supervision within the native autoregressive training objective, using losses on future video latents to train the model to encode context into more effective causal memory. Across extensive long‑horizon frame‑wise and chunk‑wise experiments under different initializations, SGF achieves stronger native long‑video extrapolation than Self Forcing, especially in subject identity, background/layout consistency, and temporal stability. Remarkably, using only a 5‑second training window, SGF can extrapolate to videos lasting several minutes. Code and models will be released to advance research on autoregressive video generation.
Authors:Changrui Zhu, Ernst Kruijff, Pengju Zhang, Simon Julier
Abstract:
We introduce MR‑Compare, a mixed reality framework for spatially grounded visual comparison between 3D Gaussian splatting and mesh reconstructions with live video see‑through (VST). Implemented on a PC‑tethered Meta Quest~3, it combines a two‑stage registration pipeline with a 3D Slider for cross‑media comparison. We evaluated five representative desktop and mobile reconstruction workflows through a real‑world benchmark with an exploratory user study (n=30) in two static indoor rooms. MR‑Compare achieved centimetre‑level translation error across all workflows. The two desktop 3DGS workflows showed the strongest overall pattern, with 3DGS‑MCMC yielding the lowest registration error and strongest VST‑referenced visual consistency. Room‑session measures indicated high perceived usability and low workload. We further propose an anisotropy filter, a zero‑shot module that leverages Gaussian anisotropies to improve 3DGS registration in MR‑Compare. A controlled Replica threshold sweep shows that moderate pruning can improve robustness and reduce residual errors. These results establish system‑level feasibility in the tested setting rather than task‑level effectiveness or standalone deployment. The project is available at https://github.com/changruizhu96/MR‑Compare.
Authors:Daniel Corva
Abstract:
Recent work established that under active inference, linear‑Gaussian state‑space models lose their epistemic drive (any incentive to act so as to gain information) "under any circumstances". The epistemic term of the Expected Free Energy becomes constant: the agent flattens to a Kalman filter whose gain sequence is fixed in advance, regardless of action. The minimal departure that restores the drive is unknown; the only established route is control entering the dynamics multiplicatively; the observation side of this boundary is unexplored. We show that state‑dependent observation noise is such a departure: a covariance R(x) that varies with the state x, representing a sensor's accuracy degrading with range. The agent runs the standard first‑order Gaussian filter of this literature, R evaluated at the predicted mean. Coupling R(x) to a controllable latent mean makes the posterior covariance, and hence the effective Kalman gain, depend on the action. Consequently, no fixed linear‑Gaussian filter reproduces the agent and, under a mild rank condition on the observation map and a non‑degeneracy condition on R(x), epistemic value is no longer constant; for scalar observations, reachable non‑constancy alone is needed. This is a minimal constructive instance of the Bar‑Shalom‑Tse dual effect in the agent's maintained covariance: actions now influence the quality of future estimates, not merely the state. Our library cpomdp detects the incompatibility from model specification alone and raises a typed IncompatibleLinearizationError. The theorem ships with an executable witness: exhibiting any fixed filter that reproduced the agent's beliefs would refute both theorem and witness at once. Together this offers a precise, observation‑side characterisation of curiosity in a Gaussian agent, bridging dual control and active inference.
Authors:Siying Wang, Kangye Ji, Di Wang, Fei Cheng
Abstract:
Diffusion policies achieve strong visuomotor control by iteratively denoising action chunks, but repeated denoising makes real‑time deployment computationally demanding. Cache‑based methods reduce inference cost by reusing intermediate activations, but existing training‑free schedules typically allocate computation uniformly across blocks, ignoring heterogeneous redundancy across blocks and leading to a suboptimal performance‑efficiency trade‑off. To bridge this gap, we introduce Evolving Cache Schedules (EVO), a training‑free acceleration framework that globally schedules cache refreshes via evolutionary search. EVO represents each candidate as a complete schedule over the block‑timestep lattice. Thus, redundant transformer computations during iterative denoising can be skipped through cache reuse while preserving closed‑loop rollout performance. To make the search practical, EVO introduces redundancy‑aware initialization, which seeds the population with promising schedules, and target‑conditioned early stopping, which verifies and terminates once a desired performance target is reached. The offline‑optimized schedule can be directly plugged into pretrained diffusion policies without retraining. Extensive manipulation benchmarks show that EVO preserves near‑full performance while substantially reducing computation, achieving up to 8.05x action‑generation speedup and reducing FLOPs from 15.77G to as low as 1.96G. Source code is available at https://github.com/pillom/EVO.
Authors:Qiwei Ma, Bin Deng, Junjie Zhu, Qiangjuan Huang, Puhong Duan, Ke Yang, Xudong Kang, Shutao Li
Abstract:
Visible‑infrared (VIS‑IR) alignment is a key pre‑training task for robust multi‑sensor perception. Most existing methods use uniform patch‑wise contrastive learning, but this can be unreliable in VIS‑IR data because imaging‑physics differences make some spatially paired regions inherently less comparable, and aligning them with equal strength hinders representation learning and downstream transfer. In this paper, we revisit VIS‑IR pre‑training from a sampling perspective and propose Importance‑Aware Sampling (IAS), which adjusts training emphasis based on patch reliability. Specifically, IAS (i) derives patch weights from infrared structural cues and uses them to reweight the contrastive objective; (ii) learns a soft importance mask with a lightweight sampler, optionally warm‑started from the hand‑crafted prior; and (iii) employs a patch curriculum learning strategy that gradually expands from high‑reliability regions to harder patches. It is worth noting that IAS is plug‑and‑play and works with both patch‑/correlation‑level alignment (e.g., UNIV‑style) and image‑level contrastive baselines (e.g., ImageBind‑style). Extensive experiments on multiple VIS‑IR benchmarks demonstrate consistent improvements over strong baselines, including for IR semantic segmentation, IR object detection and VIS semantic segmentation and cross‑modal retrieval task. Code will be released on https://github.com/KlayMa527/IAS.
Authors:Yang Xu, Gurpreet Singh Mukker, Raymond Wang, Jasper Gerigk, Maria Attarian, Igor Gilitschenski
Abstract:
Practical robotic grasping in complex scenes requires both 3D spatial reasoning and alignment with task‑specific requirements. Vision‑language models (VLMs) offer a natural way to specify these requirements using language, but existing approaches either use a VLM to predict the grasp directly with limited spatial awareness, or train the VLM together with the grasping model, which requires significantly more data and compute. These limitations impede performance and have prevented scaling to multiple embodiments in complex scenes. We address this by proposing SeededGrasp, a novel data‑efficient framework that enables a VLM to predict a seed point to be used as conditioning for a subsequent lightweight grasp‑generation model. Our architecture decouples high‑level semantic reasoning from low‑level geometric execution, enabling multi‑embodiment support while bypassing the need for expensive end‑to‑end training. To enable training such models, we release the first multi‑embodiment tabletop grasping dataset comprising over 2.5M grasps in cluttered scenes. Experimental results demonstrate that our approach outperforms existing baselines, achieving 72% success in simulation and 78% in real‑world grasping experiments. See our project site for data and code: https://uoft‑isl.github.io/seeded‑grasp/
Authors:Zejing Rao, Haoxian Zhang, Xiaoqiang Liu, Yiping Meng, Guoxin Zhang, Pengfei Wan, Fan Tang, Tong-Yee Lee
Abstract:
Existing human‑‑object interaction (HOI) video generation methods are largely limited to offline short‑video generation with complex driving conditions, making them unsuitable for real‑time interactive applications. We present \emphStreamHOI, a low‑latency streaming framework for long‑duration HOI video generation. Instead of converting heavily conditioned HOI pipelines into streaming systems, we study how an image‑to‑video streaming generator should organize historical memory to preserve interactions under bounded latency. We find that the standard sink‑local memory design faces a trade‑off in streaming HOI generation, and different transformer blocks show different historical‑memory preferences for HOI regions and surrounding regions. To match memory composition with block behavior, StreamHOI performs offline HOI‑aware block profiling and applies bias‑guided memory‑specialized training to adapt the generator to block‑specific memory layouts. We further introduce a memory distance scaling module to strengthen long‑range access to early interaction states. Extensive comparisons with both long‑video baselines and recent HOI generation methods demonstrate that StreamHOI achieves strong interaction plausibility, object fidelity, human quality and efficiency, reaching 17.6 FPS with 0.75s first‑chunk latency.
Authors:Vitor M. Leitao, Juscimara G. Avelino, George D. C. Cavalcanti, Rafael M. O. Cruz
Abstract:
Imbalanced regression problems arise when the target variable has an asymmetric distribution, resulting in underrepresented value ranges in the dataset. Traditional approaches for identifying rare instances rely on a relevance function that assigns higher importance to specific regions of the target distribution. However, the effectiveness of imbalance‑aware learning methods depends strongly on how relevance is defined. In more complex scenarios, such as bimodal distributions, traditional relevance functions struggle to capture rarity, as they assign fixed relevance values based solely on target values, thereby compromising the distinction between truly rare and normal instances. To address these limitations, this study proposes an Instance Hardness‑based relevance function (InHaR) for identifying rare instances in regression problems. Unlike traditional relevance functions, the proposed approach incorporates learning difficulty, allowing rarity to be inferred not only from the target distribution but also from the difficulty of instances for the learning algorithm. This property is particularly important in bimodal scenarios, where rarity cannot be accurately inferred from target values alone. Experimental results demonstrate that the InHaR correctly identifies rare regions under bimodal distributions and, when used to guide resampling strategies such as Random Oversampling (RO) and Gaussian Noise (GN), leads to significant improvements in predictive performance compared to traditional relevance‑based approaches. The code, dataset, and further details about the proposed method are publicly available at https://github.com/VitorLeitao/instance‑hardness‑Imbalanced‑regression.
Authors:Sriprabha Ramanarayanan, Rahul G. S., Mohammad Al Fahim, Keerthi Ram, Ramesh Venkatesan, Mohanasankar Sivaprakasam
Abstract:
Attention Mechanism (AM) selectively focuses on essential information for imaging tasks and captures relationships between distant pixel neighborhoods to compute feature representations. Accelerated MRI reconstruction benefits from AM, as the imaging process involves Fourier domain measurements that influence image representation non‑locally. However, AM‑based models are more adept at capturing low‑frequency information with limited capacity for high‑frequency representations, restricting models to smooth reconstruction. Additionally, AM‑based models need mode‑specific retraining for multimodal MRI data, as their knowledge is restricted to local contextual variations that may be inadequate to capture transferable features across heterogeneous domains. To address these challenges, we propose a neuromodulation‑based discriminative multi‑spectral AM for scalable MRI reconstruction that can (i) propagate context‑aware high‑frequency details for high‑quality reconstruction, and (ii) capture features reusable across deviated unseen domains in multimodal MRI. The proposed network consists of a spectral filtering CNN to capture mode‑specific transferable features and a dynamic high‑pass kernel generation transformer focusing on high‑frequency details. We evaluate our model on comparative studies in supervised and self‑supervised learning, diffusion model‑based training, closed‑set and open‑set generalization under heterogeneous MRI data, and interpretation‑based analysis. Our method offers scalable, high‑quality reconstruction with best improvement margins of ~1 dB in PSNR and ~0.01 in SSIM under unseen scenarios. Code: https://github.com/sriprabhar/SHFormer
Authors:Juscimara G. Avelino, Juscelino S. A. Junior, George D. C. Cavalcanti, Rafael M. O. Cruz
Abstract:
Cross‑Project Defect Prediction (CPDP) involves building models using data from external projects, called training projects, to predict modules from the target project. However, traditional CPDP methods suffer from the distribution shift between training and target projects that affects the model's performance. This paper proposes a novel CPDP framework that addresses this issue by proposing a two‑stage multiple classifier system (MCS) selection scheme: one working at the project level and another at the module level. In the first stage, the framework evaluates multiple possible MCS configurations to find one that covers and generalizes well across multiple training projects. Consequently, the proposal is likely to obtain a diverse set of classifiers, each specialized in tackling software modules with distinct characteristics. The second selection stage operates at test time, selecting the most competent classifiers to predict each new module in the target project. Unlike previous approaches that apply the same classifiers to the entire target project, the proposed framework performs module‑level model selection. This way, the system is more robust to changes in distributions between training and target projects because the selected set of classifiers is module‑dependent. Our experimental results using 82 projects from four different CPDP benchmark datasets demonstrate that the proposed approach outperforms the state‑of‑the‑art CPDP methods in most scenarios. The code, dataset, and further details about the proposed method are publicly available at https://github.com/jsaj/Multi_DES.
Authors:Jinliang Shen, Lianghao Su, Zheming Li, Kang He, ZiLiang Lai, Yanbing Jiang, Chengru Song
Abstract:
Autoregressive (AR) video diffusion models have become a promising paradigm for long and streaming video synthesis, but the continuously growing Key‑Value (KV) cache makes attention the dominant inference cost, especially at high resolution where each frame contributes many tokens. Existing remedies either evict the cache with coarse heuristics that cause inter‑frame flickering, or require model re‑training. We propose HeadCast, a training‑free, plug‑and‑play acceleration framework built on the observation that a pre‑trained AR model's attention heads exhibit stable, heterogeneous behaviors. After a short warm‑up, HeadCast performs a one‑time classification at the maximum‑noise step that sorts every head into one of four archetypes: Sink, Dummy, Spatial, and Global, and restructures the monolithic KV cache into head‑specific pathways. Crucially, it retains the Global heads that preserve the long‑range temporal consistency aggressive eviction destroys. Because the Spatial pathway operates on a fixed‑size grid, its savings grow with resolution: across state‑of‑the‑art AR models, HeadCast accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full attention and largely flicker‑free. Code is available at https://github.com/sjlgaga/HeadCast .
Authors:Sina Amirrajab, Volker Vehof, Michael Bietenbeck, Nuriye Akyol, Redouane Bouras, Khuraman Isgandarova, Alexandru Zlibut, Philipp Stalling, Ali Yilmaz
Abstract:
Aims: Cardiovascular magnetic resonance (CMR) imaging enables non‑invasive assessment of myocardial structure, function, and pathology, but requires substantial experience in interpretation of CMR images that could be supported by artificial intelligence (AI)‑based models. However, use of AI models for enhanced CMR reading is limited by labor‑intensive data curation, suboptimal model performance, and unclear implementation pathways. Methods and results: We developed an automated data curation pipeline for CMR‑based cardiovascular disease (CVD) diagnosis, integrating open‑source locally‑run large language models (LLMs) to extract diagnostic labels from narrative CMR reports and preprocessing multimodal imaging data, including cine and late‑gadolinium‑enhancement (LGE) CMR sequences. Three vision foundation models (DINO, VST, UMedPT) were fine‑tuned across these modalities in a two‑stage approach. The dataset comprised hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), ischemic cardiomyopathy (ICM), cardiac amyloidosis (CA), and normal controls (NOR). A total of 988 curated cases were randomly divided into 742 for training and 246 for validation. Fine‑tuned AI‑models achieved high discriminative diagnostic performance on an independent test set comprising 1067 patients , with individual AUC‑ROC values of up to 0.937 for the correct diagnosis of HCM and 0.945 for cardiac amyloidosis. Ensemble strategies combining multiple models and modalities further improved AI‑based diagnostic accuracy and robustness, achieving the highest overall diagnostic performance for HCM (AUC=0.959, CI [0.936‑0.978]), CA (AUC=0.966, CI [0.939‑0.986]), NOR (AUC=0.872, CI [0.852‑0.894]), DCM (AUC=0.848, CI [0.808‑0.885]) and ICM (AUC=0.840, CI [0.809‑0.868]). All training and inference code, along with the trained model weights, are publicly available on https://github.com/sinaamirrajab/CMR_CVD.
Authors:Alexis Fox, Junlin Wang, Paul Rosu, Bhuwan Dhingra
Abstract:
Long‑horizon tasks require sustained perception, reasoning, and exploration, and are a persistent challenge for large language model (LLM) agents. This gap is reflected in their limited performance on continual learning benchmarks such as ARC‑AGI‑3, especially when models are evaluated out of the box. Various agent harnesses have been proposed to close this gap, and each commits to a strategy for handling long sequences of observations, i.e., what information to save from the environment and how to load it into model context, a choice we argue is particularly consequential. Existing methods for context management face a significant tradeoff, as preserving more information makes retrieving relevant details less tractable. We propose PRO‑LONG, a minimal context management framework built around programmatic memory for LLM agents in long‑horizon, exploratory settings. PRO‑LONG addresses the tradeoff by keeping a complete, structured interaction log and capitalizing on recent progress in coding agents to search this history efficiently. On the full ARC‑AGI‑3 public game set, PRO‑LONG improves over a base coding agent by an average of 18.0 percentage points across frontier models, and matches or exceeds state‑of‑the‑art specialized harnesses (up to 76.1% pass@1) while using 4.2‑5.8x fewer tokens. With Fable 5, PRO‑LONG achieves 97.4% best@2 at a total cost of \1,750. Relevant code and logs are available at https://github.com/alexisfox7/PRO‑LONG.
Authors:Hanjing Ye, Tianle Zeng, Jiazhao Zhang, Shaoan Wang, Zibo Zhang, Weisi Situ, Yuchen Zhou, Yonggen Ling, Hong Zhang
Abstract:
Embodied visual tracking (EVT) requires a mobile agent to continuously follow a specific target described in natural language using only onboard vision. While recent vision‑language‑action (VLA) policies unify target identification and trajectory planning, their chain‑of‑thought (CoT) reasoning often operates in abstract spatial latents that are difficult to supervise and weakly aligned with explicit image‑space detections. To address this, we introduce ReferTrack, a referring‑then‑tracking paradigm that grounds EVT using a single forward‑facing camera. Our model first selects the target from an indexed set of bounding boxes, then decodes tracking waypoints conditioned on this image‑grounded decision. To preserve target motion cues over time, ReferTrack maintains a sliding‑window queue of previously selected bounding boxes, injecting their geometric features into the visual history via temporal‑viewpoint‑bbox indicator (TVBI) tokens. We further enhance target identification by co‑training on a custom Refer‑QA dataset. On EVT‑Bench, ReferTrack achieves state‑of‑the‑art single‑view performance with success rates of 89.4%, 73.3%, and 74.1% on the single‑target, distracted, and ambiguity tracking splits, respectively ‑‑ matching or even surpassing several multi‑camera baselines on identification‑heavy tasks. Finally, real‑world deployments on legged and humanoid robots validate its robust sim‑to‑real transfer capabilities. Code is available at https://github.com/MedlarTea/referTrack.
Authors:Xiaoliang Shi, Zichen Wang, Runze Ma, Zhongyue Zhang, Shuangjia Zheng
Abstract:
Antibodies are essential proteins that play a central role in immune recognition by binding specific antigen molecules. Although recent protein language models have enabled progress in single‑chain protein modeling and generation, they often fall short in antigen‑specific antibody design, where effective modeling requires explicit pairing between antibody and antigen, particularly at the epitope level. To address these limitations, we introduce AAMFM, an Antigen‑specific Antibody Multimodal Foundation Model that learns unified representations of antibody sequences and structures conditioned on antigen context. AAMFM incorporates rich antigen information including geometric interfaces and epitope annotations via a cross‑modal adapter, enabling joint modeling of antibody‑antigen interactions in a shared latent space. To further guide the model toward functional relevance, we fine‑tune AAMFM using Calibrated Direct Preference Optimization (Cal‑DPO), leveraging preference signals extracted from a strong structural prior to align learning with binding‑specific objectives. Extensive experiments demonstrate that AAMFM achieves state‑of‑the‑art performance in functional antibody design, revealing its potential for antigen‑specific antibody engineering. Our code is available at https://github.com/XL‑S224/AAMFM.
Authors:Oliver Mills, Philip Conaghan, Samuel Relton
Abstract:
Robust out‑of‑the‑box performance is essential for the clinical deployment of deep learning models in medical imaging. An important but underexplored factor affecting model generalisability is intensity normalisation, particularly for magnetic resonance imaging (MRI), where image intensities vary across scanners and protocols. In this study, we systematically compared seven normalisation methods and their impact on the performance of a 3D U‑Net model for meniscus segmentation from knee MRI. The methods included standard scaling approaches, histogram‑based techniques, and a Gaussian Mixture Model (GMM)‑based method. Models were trained on the IWOAI 2019 dataset and evaluated on both internal and external test sets (SKM‑TEA) to assess generalisability. Performance was similar internally but differences were significant on external data, with Z‑score, Nyúl histogram matching, and CLAHE showing greater robustness than other methods. However, these differences were small compared to the significant performance drop observed between datasets. Overall, while intensity normalisation had a measurable effect on model generalisability, its impact was limited relative to the effects of domain shift, highlighting the need for complementary strategies for robust deployment.
Authors:Luis Gasco, Hermenegildo Fabregat, Laura García-Sardiña, Paula Estrella, Casimiro Pio Carrino, Daniel Deniz, Alvaro Rodrigo, Rabih Zbib
Abstract:
This paper presents the second edition of the TalentCLEF Challenge, which will run as an evaluation lab as part of CLEF 2026. The aim of TalentCLEF is to promote the development of systems and methods that use Natural Language Processing (NLP) in the field of Human Capital Management (HCM), fostering approaches that ensure fairness in results, operate across multiple languages, and adapt to diverse industries. To this end, TalentCLEF establishes public benchmarks where research teams can compare methods and share findings, moving the field toward more practical and impactful NLP solutions that effectively address the real needs of workforce management. This year's lab will feature two tasks designed to foster the development and evaluation of systems that support key HCM activities such as talent matching, upskilling, reskilling, and skill gap detection: (i) Task A ‑ Contextualized Job‑Person Matching, focused on retrieving and ranking suitable candidates for specific job positions using context‑rich and privacy‑preserving data; and (ii) Task B ‑ Job‑Skill Matching with Skill Type Classification, centered on identifying relevant skills for a given job title and classifying them by their type within the job profile. TalentCLEF website: https://talentclef.github.io/talentclef/
Authors:Mark Schutera
Abstract:
tiny_schiller closes the small‑language‑model prototyping, fine‑tuning, education, and research gap for German literary text, providing a single‑file, drop‑in counterpart to Karpathy's tiny_shakespeare. The available German literary corpora are larger and richer, but require parser engineering before a single line of training or fine‑tuning code can run. tiny_schiller is a 2.07‑megabyte single file of eleven public‑domain Schiller dramas, sourced from DraCor's GerDraCor export (CC0) and processed by deterministic parser engineering. Character‑level, GPT‑2 byte‑pair encoding, and cl100k_base tokenization splits, an instruction‑formatted dialogue‑completion split, and 89 per‑character persona splits load from a single HuggingFace call. A small language model literally reaches German literary text in one line of code.
Authors:Honghao Li, Xianquan Wang, Zibin Zhang, Yi Zhang, Kangyi Lin, Yiwen Zhang
Abstract:
Ranking is a core stage in online advertising and recommender systems. Modern ranking models increasingly unify sequential modeling and feature interaction, yet many advances rely on proprietary data, closed implementations, and large‑scale industrial infrastructure. This setting limits reproducible comparison and hinders academic study of scaling laws, long‑sequence modeling, and multi‑task ranking. To address these limitations, this paper proposes UniRank, an open benchmark for ranking models that unify sequential modeling and feature interaction. UniRank uses chronological pointwise autoregressive supervision, standardizes evaluation across feedback tasks, and provides a PyTorch toolkit with Distributed Data Parallel training, operator optimization, mixed‑precision training, attention optimization, and other efficiency techniques that reduce hardware requirements. We benchmark 15 representative unified ranking models on five large‑scale public datasets from short‑video, advertising, and e‑commerce platforms, with the largest dataset containing over 700 million instances and the longest behavior sequence exceeding 10^5 interactions. UniRank provides a reproducible basis for comparing unified ranking models, studying scaling laws under limited compute, and narrowing the gap between academic and industrial ranking research. We believe UniRank benefits researchers, practitioners, and beginners through reproducible experiments, production‑oriented evaluation, and accessible implementations. Code and data are available at https://github.com/salmon1802/UniRank.
Authors:Qian Qiao, Wenye Liu, Ting Liu, Jiuhe Shu, Peng Wang
Abstract:
Cross‑view geo‑localization between UAV and satellite imagery remains a fundamental yet highly challenging task, especially under large off‑nadir views where drastic perspective distortions, occlusions, and appearance gaps occur. Existing benchmarks and methods primarily focus on near‑nadir scenarios and often overlook the importance of structural scene understanding and intra‑domain relational constraints, limiting their performance in real‑world deployments. In this work, we introduce OffNadirLoc, a new benchmark for large off‑nadir UAV‑to‑satellite geo‑localization. To tackle the unique challenges posed by off‑nadir perspectives, we further propose ONLoc, a framework that incorporates a structure‑aware contextual weighting mechanism to dynamically emphasize reliable local features while suppressing ambiguous or repetitive regions. Additionally, we design a view‑coherent learning strategy, which treats one satellite image and the corresponding UAV images from multiple views as a cohesive semantic group. This set‑level supervision enables the model to learn viewpoint‑invariant and discriminative features, making it more effective at capturing multi‑view consistency than conventional pairwise contrastive learning. Extensive experiments on the OffNadirLoc benchmark and four near‑nadir datasets demonstrate that our method consistently outperforms state‑of‑the‑art approaches while exhibiting strong zero‑shot generalization to unseen datasets without additional training. The code will be released at https://montalario.github.io/offnadirloc/.
Authors:Xinghao Guo, Yin Xu, Dazhi He, Hanjiang Hong, Zhiyong Chen, Cixiao Zhang, Yiyan Wu, Wenjun Zhang
Abstract:
Current digital semantic communication systems have primarily focused on maintaining compatibility with conventional constellation‑based modulation. In contrast, index modulation (IM) represents a more spectrally and energy‑efficient alternative by exploiting additional dimensions for information conveyance. Recognizing this potential, this paper bridges the gap between IM and semantic communications by proposing a novel spatial semantic communication (SSC) system leveraging cutting‑edge fluid antenna‑IM (FA‑IM) technology. Compatible with existing joint source‑channel coding (JSCC) architectures, the proposed SSC system employs the residual quantization (RQ) approach to discretize analog semantic features for subsequent digital IM transmission. Notably, the proposed SSC system synergizes RQ and IM via a semantic‑aware stream splitting scheme, which ensures that critical semantic information undergoes less severe channel fading, thereby further optimizing semantic transmission performance. Simulation results validate that the proposed SSC system effectively integrates the high fidelity of RQ, the reliability of semantic‑aware splitting, and the spatial efficiency of FA‑IM, thereby providing a robust solution for future digital semantic transmission. The open source code is available at: https://github.com/gxh1106/SSC.
Authors:Seonsoo Kim, Seongil Hong, Jun-Gill Kang
Abstract:
We propose Diffusion ReRoll, a diffusion‑based framework for robotic sequential prediction that enables revisable denoising over horizons. Existing diffusion‑based sequence predictors typically perform a single monotonic denoising process. In contrast, Diffusion ReRoll selectively re‑noises regions that have become locally stable while the remaining regions continue denoising, so the re‑noised regions can be refined again using context from the rest of the horizon. This structured re‑noising enables iterative cross‑horizon revision, allowing earlier and later segments to revise one another, while maintaining local consistency. We evaluate Diffusion ReRoll against full‑sequence diffusion and causal denoising based on Diffusion Forcing across long‑horizon planning, policy learning, and unified video‑action modeling. On OGBench PointMaze and AntMaze, Diffusion ReRoll achieves relative gains in average success rate of 21% over Diffusion Forcing in matched guidance‑based planning and 23% over Diffuser in matched goal‑inpainting. In diffusion‑policy‑style action prediction, Diffusion ReRoll improves average success by 56.5% relative to Diffusion Policy across different prediction horizons and history lengths on the LIBERO‑10 multi‑task benchmark. In unified video‑action prediction, Diffusion ReRoll improves policy and inverse dynamics performance, especially under out‑of‑distribution evaluation, and achieves the best action‑video consistency. These results support structured re‑noising as an effective mechanism for revisable robotic sequence generation.
Authors:Yufan Zhu, Kefu Yi, Xueju Zhang, Yunyang Tian, Long Chen, Zixuan Xiao
Abstract:
Long‑range vehicle trajectories provide important spatio‑temporal evidence for traffic safety analysis, autonomous driving evaluation, and data‑driven traffic management, yet continuously recovering them from fixed highway cameras remains difficult. As vehicles recede into distant road regions, perspective compression and scale decay often fragment or prematurely terminate automatic tracklets, even when their continuation remains identifiable from motion consistency across neighboring frames. We formulate this problem as recovering the far‑range continuation of a vehicle trajectory from a reliable near‑field tracklet. We introduce LoRFT, to our knowledge the first open benchmark dedicated to long‑range vehicle trajectory reconstruction from fixed highway cameras. LoRFT comprises 22 expressway surveillance scenes, 366,109 video frames, 6,601 manually verified trajectories, 2,694,889 bounding boxes, road‑geometry annotations, scene‑level splits, and evaluation scripts. We further propose Map‑RSTNet, a map‑aware residual sequence‑to‑sequence model that reconstructs distant trajectories in a road‑geometry‑aligned state space and dynamically refreshes local road geometry during decoding. On LoRFT, Map‑RSTNet reduces ADE, FDE, and 5‑second RMSE by 11.0%, 15.4%, and 10.5%, respectively, relative to the strongest baseline. These results demonstrate that road‑geometry‑aware reconstruction can extend usable trajectory records from existing fixed‑camera infrastructure. LoRFT provides a reproducible testbed for long‑range vehicle trajectory reconstruction.
Authors:Habin Lim, Gyeong-Moon Park
Abstract:
Text‑guided video editing with diffusion models is impractically slow, hindered by costly multi‑step sampling and inversion. We present OSVE, the first framework to successfully adapt one‑step Text‑to‑Image (T2I) models for high‑quality video editing, addressing the core challenges of inversion, editability, and temporal consistency. To bypass slow iterative inversion, we train a learnable encoder that predicts the initial noise for each frame in a single forward pass. This encoder is trained with a novel Structure‑Aware Editing (SAE) loss on a curated dataset of structurally‑aligned image pairs, teaching it to preserve the source video's geometry during edits. For temporal coherence, we introduce Unified‑Frame Editing (UFE), a technique that concatenates frame latents to facilitate cross‑frame attention in a single generation step. Furthermore, for long videos, a sliding‑window strategy with an anchor frame maintains global consistency. Our extensive experiments demonstrate that OSVE achieves editing quality comparable or superior to state‑of‑the‑art multi‑step methods, while operating approximately 155‑‑171 times faster. This breakthrough paves the way for practical, real‑time video editing applications. Code is available at https://github.com/KU‑VGI/OSVE.
Authors:Qihang Wu, Austin Rovinski
Abstract:
Priority queues ‑ data structures that serve elements based on priority rather than insertion order ‑ are fundamental in a wide range of applications, including operating systems, graph algorithms, and data compression. Software implementations, typically based on binary heaps with O(log N) complexity, are sufficient for many scenarios; however they can become performance bottlenecks in latency‑sensitive domains such as networking and robotics. Hardware‑based priority queues exploit parallelism to significantly reduce operation latency, delivering critical performance improvements in latency‑sensitive applications. Despite the breadth of prior work on hardware priority queues, two major challenges remain. First, many foundational architectures were proposed and studied years ago, calling into question their relevance given modern hardware advancements. Second, comprehensive comparisons across different architectures are lacking, making it difficult to evaluate trade‑offs in performance, resource utilization, and scalability. This paper addresses both gaps by implementing and evaluating several representative hardware priority queue architectures on modern FPGA platforms and providing a quantitative analysis to guide future design choices. All implementations, tests, and analyses are available through our open‑source library at https://github.com/realise‑lab/hwpq.
Authors:Cantao Su, Menan Velayuthan, Esther Ploeger, Dong Nguyen, Anna Wegmann
Abstract:
There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmented: While there exist a number of tools for measuring the lexical diversity of texts, researchers lack standardized tools for quantifying diversity based on embeddings. Embedding‑based diversity measures are highly flexible: They work with any embedding model and any data that can be embedded, and are thus applicable to many notions of diversity. With emb‑diversity, we provide a comprehensive embedding‑based diversity measurement tool, spanning a broad range of measures. We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets. https://github.com/nlpsoc/emb‑diversity/
Authors:Yurong Liu, Yeye He, Haoyu Dong, Junjie Xing, Shi Han, Dongmei Zhang, Surajit Chaudhuri
Abstract:
Predicting missing cell values in tabular data is a fundamental problem in data cleaning. While state‑of‑the‑art reasoning models show great promise in predicting missing values in tables, by reasoning holistically across rows and columns, they are costly to deploy at scale and tend to be overconfident, often generating hallucinated or false‑positive predictions. In this paper, we observe that achieving high‑precision missing‑value prediction in tables requires a distinct combination of three capabilities: (1) world knowledge, (2) text‑based reasoning, and (3) code‑based reasoning. We systematically explore design choices for combining these capabilities, and propose an Auto‑Fill approach that post‑trains three specialist small language models (SLMs), each optimized for one capability. We develop a calibrated ensemble mechanism that either dynamically selects the most confident specialist or abstains, ensuring high accuracy. Extensive experiments on 11 benchmarks with 2200 real tables drawn from diverse domains show that Auto‑Fill achieves superior accuracy compared to state‑of‑the‑art reasoning models (e.g., o3‑pro, Gemini 3 Pro, and DeepSeek R1), while operating at a fraction (less than 1%) of the cost of these frontier models. Our results highlight the effectiveness of specialization and calibrated abstention in the important domain of tabular data. Auto‑Fill is publicly available at https://github.com/lyrain2001/auto‑fill.
Authors:Siyi Hao, Yidi Cao, Linhao Yu, Yuqi Ren, Deyi Xiong
Abstract:
With the wide application of large language models (LLMs) in real‑world scenarios, the value implication of their outputs is crucial. However, existing evaluation benchmarks suffer from insufficient coverage of value dilemmas in daily scenarios involving multiple value conflicts and simplistic evaluation formalisms that fail to assess LLMs' value alignment. To address these issues, we propose D2VBench, a value alignment benchmark comprising 10,000 instances of real daily dilemma scenarios constructed through a multi‑stage collaboration between LLMs and humans, grounded in 158 manually annotated fine‑grained value concepts. For evaluation on the benchmark, we present a hybrid evaluation paradigm that integrates multiple‑choice questions with open‑ended questions. We conduct comprehensive evaluations on eight mainstream LLMs. Experimental results demonstrate that D2VBench exhibits high reliability and robustness, effectively reflecting the LLMs' alignment across different value categories and dimensions, and providing a more realistic and fine‑grained tool for research on value alignment. The dataset is available at https://github.com/tjunlp‑lab/D2VBench.
Authors:Huangbiao Xu, Huanqi Wu, Xiao Ke, Jiaxin Cai, Junyi Wu, Jinglin Xu
Abstract:
Action Quality Assessment (AQA) aims to objectively evaluate performance quality from action videos. Most existing methods follow a ``one‑by‑one'' paradigm, training a separate model for each action type. This setting limits real‑world deployment, as it requires prior action‑type knowledge to select the corresponding model and suffers from poor generalization across diverse actions. To address these limitations, we study the challenging task of all‑in‑one AQA, which aims to assess heterogeneous actions within a single unified model. We propose a novel Mixture of Action Knowledge Experts (MoAKE) framework, designed to mitigate negative knowledge transfer caused by large semantic discrepancies among actions. MoAKE learns complementary experts that capture diverse action patterns within a shared semantic space and dynamically aggregates their knowledge to adapt the assessment to the input action. Each expert is tailored with segment‑aware prototypes to handle varying temporal lengths, together with an Adaptive Intra‑ and Inter‑Segment Relationship Modeling (AIISRM) module to model multi‑granularity temporal dynamics. Furthermore, we establish comprehensive benchmarks for all‑in‑one as well as zero/few‑shot AQA. Extensive experiments on three long‑term datasets demonstrate that MoAKE significantly outperforms existing methods in the all‑in‑one setting, while also achieving consistent generalization on three short‑term datasets under zero/few‑shot evaluation. Code is available at https://github.com/XuHuangbiao/MoAKE.
Authors:Xudong Ouyang, Wenlun Zhang, Yimin Xu, Huazhong Liu, Yunshan Zhong
Abstract:
The Segment Anything Model 2 (SAM2) has advanced temporal promptable segmentation, yet its deployment remains hindered by heavy memory cross‑attention overhead and redundant full‑frame visual feature extraction. While recent methods explore efficiency via heuristic memory pruning and window‑based sparse routing, they typically suffer from catastrophic performance degradation in complex segmentation scenarios replete with occlusions and distractors. To resolve these limitations, we propose Lean‑SAM2, a holistic lightweight framework designed to address the above vulnerabilities while systematically eliminating computational redundancies. Specifically, Lean‑SAM2 integrates three collaborative mechanisms: (1) Target‑Anchored Memory Pruning (TAMP) safeguards target tokens against deceptive attention by modulating raw attention significance with semantic consistency against prompt‑derived foreground anchors; (2) Temporal Condensation with Insurance Memory (TCIM) condenses historical context via a visibility‑gated fusion while conditionally archiving high‑confidence entries in a parallel insurance bank; and (3) Target‑Anchored Risk‑Aware Routing (TARR) selectively activates the heavy image encoder for target‑related windows based on anchor similarity, utilizing a risk‑aware fallback policy to trigger full‑frame refreshes during volatile transitions. Extensive evaluations across multiple challenging benchmarks demonstrate that Lean‑SAM2 establishes a superior balance between accuracy and efficiency. For example, on the LVOSv2 validation dataset, Lean‑SAM2 achieves overall inference speedups of 1.412× and 1.417× on the SAM2.1‑Large and SAM2.1‑Base+, respectively, significantly outperforming Efficient‑SAM2 while boosting the corresponding \mathcalJ\&\mathcalF scores by 5.0% and 3.6%. Code is available at https://github.com/DeawhaleQwQ/Lean‑SAM2.
Authors:Jiandong Ding, Tianying Liu, Fuyuan Liu, Huijie Qin, Tiandeng Wu
Abstract:
Sequential recommendation (SR) models capture continuously observed behavior, but a returning user may have no interactions for months or years. We define this setting as Zero‑Observation Reactivation: the user has a pre‑gap history, while the platform observes no behavioral signals during a macro‑gap Delta t. Under a chronologically aligned Gap‑Synthesize Protocol on three Amazon datasets (Video Games, CDs & Vinyl, and Movies & TV), Hit@10 decreases monotonically across the evaluated gap buckets and reaches its lowest level beyond one year. The pattern appears across recurrent, unidirectional, and bidirectional SR backbones. We propose DeltaGate, a lightweight output‑layer plugin that keeps the backbone frozen and routes each representation dimension between the personalized history and a learned, zero‑initialized global prior. The gate is conditioned jointly on Delta t and the personalized representation. In a controlled diagnostic, we hold the personalized representation fixed and vary Delta t to isolate the trained gate's response to the gap input. In the >365d Video Games bucket, DG‑SASRec reaches 0.047 Hit@10 versus 0.031 for SASRec, while DG‑BERT4Rec reaches 0.046 versus 0.025 for BERT4Rec, with 66K trainable parameters (2‑‑4% overhead). End‑to‑end retraining attains higher absolute accuracy but changes the backbone embeddings; the frozen plugin preserves zero backbone drift, uses about 40x fewer trainable parameters, and retains observable dimension‑wise routing. The source code is available at https://github.com/jdding/DeltaGate.
Authors:Zhengxian Wu, Junjie Gao, Kai Yang
Abstract:
Multimodal agentic search systems increasingly rely on external tools to answer knowledge‑intensive visual questions. However, existing evaluations mainly focus on final‑answer accuracy and may miss failures in the search trajectory. In this work, we study such hidden reliability issues as silent failures. We introduce a six‑category taxonomy covering modality shortcuts, phantom grounding, wrong‑evidence‑right‑answer cases, over‑retrieval laundering, cross‑modal contradiction, and provenance hallucination. Based on this taxonomy, we build a trajectory‑level diagnostic pipeline that evaluates both answer correctness and evidence‑grounding quality under a unified ReAct‑style scaffold. Experiments on MMSearch‑Plus trajectories across four frontier multimodal models show that surface accuracy consistently overestimates true trajectory‑level correctness. We further use cross‑judge validation, blank‑image stress tests, and tool ablations to show that silent failures are capability‑dependent and often shift rather than disappear. Home‑page: https://github.com/DingWu1021/silent‑failures‑multimodal‑agentic‑search
Authors:Md Tanvirul Alam
Abstract:
Reinforcement learning with verifiable rewards (RLVR) has substantially improved language‑model reasoning, yet its extension to vision‑language models remains constrained by the lack of training data that are simultaneously broad, exactly verifiable, and reproducible. We introduce Trace, a taxonomy‑guided environment for multidomain visual reasoning. Trace factorizes task construction into a scene grammar and an executable task program, separating visual realization from answer computation. A shared semantic state determines the rendered image, prompt, typed answer, verifier state, and replayable instance trace. The resulting environment comprises 1,000 tasks over 277 scene grammars and 11 visual domains, with controlled semantic and visual variation. RLVR on 64,000 Trace instances improves the macro‑average across 24 external benchmarks by 3.51 percentage points for Qwen2.5‑VL‑3B and 4.06 points for Qwen2.5‑VL‑7B, providing evidence that broad procedural training can transfer beyond the generated task distributions. Project page: https://maveryn.github.io/trace/.
Authors:Kabir Murjani, Mishri Bhavsar, Manish I. Patel, Jonti Talukdar
Abstract:
Very Large Scale Integration (VLSI) global routing is an NP‑hard combinatorial optimization problem requiring signal net assignment across capacity‑constrained 3D grids while minimizing congestion, wirelength, and via transitions. Because traditional heuristics rely on static penalty schedules that fail on complex congestion topologies, we present AlphaRoute: a multi‑objective adaptive search framework reformulating rip‑up and reroute (R&R) into a dynamic optimization system. We introduce SHAP‑based overflow decomposition to isolate per‑net congestion, driving targeted subgraph extraction via 3D Dijkstra maze routing and an adaptive PathFinder policy. Crucially, AlphaRoute employs Large Language Models (LLMs) as semantic policy optimizers. Bounded by a deterministic knowledge graph, the LLMs interpret congestion metrics to dynamically adjust penalty parameters. Evaluated on ISPD 2025 benchmarks, AlphaRoute reduces overflow by 98.6% on MEMPOOL. On the constrained ARIANE design, we achieve an overflow of 146,109 (a 29.8x reduction in overflow over the state of the art), yielding a penalized score of S_orig = 0.0538 versus the State‑of‑the‑art (SOTA) 1.780. These results demonstrate that superior algorithmic search geometry can overcome the latency of interpreted Python implementations.
Authors:Kwonyoung Ryu, In-Jae Lee, Jonghyun Jin, Hyunjee Lee, Jongmin Lee, Jaesik Park
Abstract:
Large view synthesis models synthesize novel views through cross‑view attention without explicit 3D representations, and recent studies have shown that they learn accurate spatial correspondence from RGB supervision alone. We observe that this correspondence generalizes beyond appearance. When non‑photorealistic signals such as binary encoded panoptic labels are passed through the model, they are propagated to novel views with consistent spatial structure. These results indicate that the correspondence learned for RGB view synthesis can also propagate view‑independent per‑pixel labels. From this observation, we present the first work to extend large view synthesis models beyond appearance rendering to 3D scene understanding. We propose a panoptic segmentation pipeline that reuses a frozen view synthesis model to propagate panoptic labels from input views to novel views, without 3D reconstruction or any segmentation‑specific training of the view synthesis model. Given panoptic labels on the input views, we encode them into binary channel representations and pass them through the same model to render target‑view segmentation. On ScanNet, our method achieves segmentation quality on par with Gaussian based approaches requiring explicit 3D reconstruction, while outperforming them in novel view synthesis by more than 7 dB. The label propagation also transfers across datasets, surpassing these approaches on Replica without any fine‑tuning.
Authors:Gurp Nijjer
Abstract:
Model‑based reinforcement‑learning agents of the DreamerV3 family forget catastrophically when trained on task sequences, even when an unbounded replay buffer preserves every earlier experience. We ask a question the continual‑RL literature has assumed an answer to but never measured: which component forgets? Under never‑clear replay, pre‑registered component‑level probes (n=3 seeds throughout) show that the world model retains essentially everything measurable about old tasks ‑‑ reward discrimination (retention ratio ~1.0), value estimates, and termination structure ‑‑ while the actor's behavior collapses. Forgetting in this regime is a channel problem, not a memory problem. We demonstrate this by intervention: with the world model frozen and identical imagined rollouts, reinforcement learning in imagination fails to recover a lost skill (0/3 seeds), while supervised self‑imitation on the world model's own graded dreams recovers it on 3/3 seeds with zero environment interaction. Interleaved during training, this graded dream rehearsal yields a task‑label‑free, parameter‑constant continual learner: 3/3 four‑task chains retained where plain replay passes 0/3, 3/3 eight‑task chains, and consistent gains over matched real‑episode cloning (paired difference +0.13, bootstrap 95% CI [0.07, 0.24], complete seed separation). The dream‑grading step is load‑bearing: we characterize two scoring failure modes, provide an offline selection gauge that caught both before they contaminated results, and give a realized‑first grading rule that closes them. All experiments were pre‑registered with committed protocols; every refuted hypothesis is reported.
Authors:Yihong Sun, Bharath Hariharan
Abstract:
Tracking objects through state transformations is essential for understanding real‑world dynamics. However, existing methods are computationally expensive. TubeletGraph recently showed impressive capabilities, but its inference cost (~4.4 seconds per object‑frame on VOST) precludes any real‑time deployment possibilities. We observe that TubeletGraph's overhead arises from building a spatiotemporal partition of the input video: (1) entity segmentation is computed densely for every frame regardless of whether a transformation occurs, and (2) every entity in the scene is tracked, scaling cost with scene complexity rather than the number of transformations of interest. To address both, we propose FluxGraph, a reactive variant that uses SAM2's internal multi‑mask disagreement as a lightweight trigger for transformation detection, and removes the need for tracking all entities in the given video. FluxGraph is ~3.3× faster than TubeletGraph on VOST while improving tracking performance and preserving state graph quality. Furthermore, we also observe consistent speedups of 3.7‑10.7× across VSCOS, M^3‑VOS, and DAVIS17 while maintaining performance. Code is publicly available at https://github.com/YihongSun/FluxGraph.
Authors:Da Li, Chang Ma, Dongfu Yin
Abstract:
Noisy and corrupted points can substantially degrade point cloud recognition performance, especially under challenging corruption settings. In particular, full fine‑tuning of 3D pre‑trained models may amplify the influence of outliers and overwrite robustness priors learned during pre‑training, while naive parameter‑efficient adaptation remains sensitive to corrupted tokens. To address this issue, we propose PSFT, a point‑selection fine‑tuning framework that improves robustness while remaining parameter‑efficient. PSFT first estimates point‑wise influence from pre‑pooling features and adaptively retains minimally influential points to suppress outliers. Based on the selected subset, a prompt generation branch predicts layer‑wise prompt tokens and injects them into a frozen backbone for lightweight downstream adaptation. To further mitigate residual noise after selection, we append a lightweight feature filter with bottleneck MLP transformation and Beta‑gated residual blending to refine patch‑token representations before prediction. Extensive experiments show that PSFT consistently reduces corruption error on ModelNet‑C and ModelNet40‑C across all tested 3D pre‑trained backbones, while achieving the strongest ScanObjectNN‑C results with ULIP‑2 and Uni3D‑B among the evaluated tuning strategies. Our implementation can be found at https://github.com/CVChMA/PSFT/tree/master.
Authors:Yingxin Liang
Abstract:
AI‑generated covers often fail through local musical errors that a global quality score cannot locate: the vocal contour may remain recognizable while the accompaniment uses the wrong harmonic function, or the output may stay in key while the arrangement remains incomplete. We present a five‑dimensional diagnostic framework covering melodic pitch, harmonic progression, key consistency, style consistency, and arrangement/production quality. The benchmark contains 30 covers generated from 5 source songs by 6 systems, with expert severity ratings and 9 symbolic or acoustic features. Harmonic progression and arrangement had the highest severe‑error rates (53% and 47%), whereas key consistency was better preserved. Six covers combined acceptable key consistency with severe harmonic errors. Large‑leap ratio had a nominal association with melodic ratings (Spearman rho = ‑0.429, uncorrected p = 0.018), but no feature correlation survived the nine‑test multiplicity reference. An interpretable percentile‑rule pilot likewise failed to outperform a fixed majority baseline reliably across 16 dimension‑level comparisons. The results separate useful diagnostic evidence from dependable automatic scoring: low‑level and symbolic summaries can expose particular symptoms, but they do not replace context‑aware musical judgment.
Authors:Li Zeng, Zeyu Ye, Meng Xie, Hangtao Zhang, Xianlong Wang, Yanchun Li, Zhetao Li
Abstract:
Vision‑Language Models (VLMs) are known to be vulnerable to adversarial attacks, where subtle perturbations to images or texts induce erroneous outputs. However, most text‑based attacks are adapted from language‑model‑centric methods, in which the visual input is fixed during optimization, resulting in adversarial prompts that are tied to specific images and thus limiting their attack effectiveness. To this end, we first introduce a new research perspective: cross‑image transferability for adversarial prompts. We then propose GhostPrompt, an adversarial prompt that is optimized once and reused to steer VLM outputs toward attacker‑specified responses across diverse images. GhostPrompt employs a joint optimization that distills image‑invariant adversarial features into the prompt by "worst‑case" generation. Specifically, it alternates between constructing hard visual conditions for the current prompt and updating the prompt to remain effective under these conditions. Extensive experiments on prevalent VLMs verify that \ourmethod achieves an improvement of over 30% in attack success rates compared to state‑of‑the‑art (SoTA) baselines, while reducing computation time by ~70%. Our code is avalable at https://github.com/Ye‑ze‑yu/GhostPrompt.
Authors:Aileen Liao, Rachel Holladay, Dinesh Jayaraman, Michael Posa
Abstract:
Despite recent advances in general‑purpose robotic manipulation, real‑world multi‑object clutter remains challenging to handle for today's prevalent approaches. The problem scales in complexity due to more objects and collisions, more unpredictable contact physics, distractors, and task ambiguity. Bridging this gap to real‑world deployment requires effective scene abstractions; yet today, producing such abstractions requires extensive task‑specific manual engineering, which does not scale. These abstractions are costly to generate and difficult to adjust or fine‑tune. We instead propose a plug‑and‑play fix to automatically generate scene‑specific, task‑specific, adaptively updating abstractions on top of existing planning and control stacks. LLM‑guided Environment Simplification (LENS) produces a de‑cluttered abstracted scene representation by merging (e.g., stacked objects) or pruning (e.g., distant objects) scene entities in a closed loop in response to task progress. These dynamic, task‑relevant abstractions are versatile and easy to use. In our experiments, we show that LENS improves classical planning, model‑based control, and a vision‑language‑action model, across a diverse set of highly cluttered manipulation scenes. Project website: https://lens‑2026.github.io/.
Authors:Kiyan Rezaee, Morteza Ziabakhsh, Artin Bahrampour, Seyed Mohammad Ghoreishi, Asal Khaje, Ali Sajedifar, Manny Chalak, Ava Zerafatangiz, Sadegh Eskandari
Abstract:
In this paper, we present SCPP (Soft Clustering Python Package), an open‑source Python framework for soft clustering. SCPP establishes a canonical, scikit‑learn‑compatible estimator interface that standardizes model training, prediction, membership representation, evaluation, and benchmarking across heterogeneous soft clustering methods, including fuzzy, probabilistic, graph‑based, matrix factorization, and deep learning methods. The framework currently integrates 40 representative algorithms together with a comprehensive benchmarking comprising datasets, clustering quality metrics, and standardized runtime, memory, and scalability evaluation. SCPP further provides extensive documentation, practical examples, automated testing, and seamless integration with the scientific Python ecosystem, enabling reproducible experimentation and straightforward extension with new algorithms. The source code is publicly available at https://github.com/soft‑clustering/soft‑clustering.
Authors:Manuel Pfeuffer, Roshan Prakash Rane, Hadya Yassin, Kerstin Ritter, Sonja Greven
Abstract:
The shape of a planar curve is the geometric information that remains once translation, rotation, scale and reparametrisation are removed and is of interest in many health applications, e.g. in neuroimaging. We propose a deep shape regression model for open planar curves that admits multimodal and high‑dimensional covariates. Representing curves as complex‑valued functions, we show that the conditional full Procrustes mean is the leading eigenfunction of the conditional covariance. To estimate this covariance surface, we propose a novel deep conditional covariance smoother with modality‑specific encoders ‑ e.g. splines for scalar covariates and convolutional networks for images, which classical spline smoothers cannot accommodate. Our model is by construction invariant to the translation, rotation and scaling of the input curves and handles sparsely and irregularly sampled curves. We further provide an algorithm for elastic mean estimation that also removes parametrisation by iterating covariance smoothing, rotational alignment and parametrisation alignment. We illustrate the method on simulated outlines with known conditional mean and multimodal covariates, and give a first application to hippocampal outlines from the ADNI cohort, recovering covariate effects consistent with the literature. Code is available at https://github.com/mpff/dnn‑shapes.
Authors:Xuefei Julie Wang, Lauren Hyoseo Yoon, Chengrui Qu, Amanda Zichang Wang, Atharva Sehgal, Eric Mazumdar, Yisong Yue
Abstract:
Self‑improving AI systems typically treat the agent as the object that improves, by optimizing prompts, workflows, harnesses, or even the agent's own code. This agent‑centric view can make improvements expensive to maintain and difficult to transfer, because gains become tied to a particular agent design, task distribution, or adaptation run. We study a complementary paradigm: knowledge‑centric self‑improvement, in which agents remain generic and disposable while the persistent object is a curated knowledge base that agents can leverage for future tasks. We conduct controlled case studies to operationalize this idea via a simple protocol. Agents attempt one task, then contribute evidence‑grounded insights to a shared knowledge base via task‑level and cross‑task forums, followed by knowledge distillation. Because self‑improvement is contained in the knowledge rather than the agent, improvement can be more inspectable, transferable, and portable. Across abstract reasoning, coding, and terminal benchmarks, this protocol improves solve rates while reducing dollar cost relative to agent‑centric baselines. The resulting distilled knowledge also transfers to held‑out tasks and across LLM families, indicating that the improvement is not merely an LLM‑ or run‑specific behavior. These results support a new view of self‑improving agentic systems: progress can be driven primarily by the curated persistent knowledge. Code is available at https://github.com/recursive‑knowledge/KSI.
Authors:Yiwei Zhou, Ziheng Chen
Abstract:
We introduce RELTA‑SGLD, a taming scheme that stabilizes superlinear stochastic‑gradient updates while reducing unnecessary suppression of the original learning drift. A threshold determines where the taming turns on, while a relative‑growth principle derived from the one‑step Lyapunov stability condition determines the required taming strength. Together, they produce a lighter λ‑scale denominator and preserve a nonvanishing far‑tail return. As a consequence, we prove polynomial moment stability and first‑order stationary accuracy in both W_1 and W_2 for nonconvex SGLD with superlinearly growing stochastic‑gradient oracles, improving the corresponding half‑order and quarter‑order bounds for comparable stochastic‑gradient tamed schemes. On Fashion‑MNIST under active stabilization pressure, RELTA improves the mean learning metrics over both untamed SGLD and TUSLA and remains competitive with a tuned AdamW reference. In an ordinary‑training regime, its lighter localized denominator reduces unnecessary perturbation of the original update and maintains nearly untamed learning dynamics.
Authors:Florian Golemo, Joanna Wolski, Joel Ruben Antony Moniz, Christopher Pal
Abstract:
Many Blind and Low‑Vision (BLV) people rely on guide dogs for moment‑to‑moment navigation, such as staying on path and avoiding obstacles and pedestrians. However, guide dogs are expensive to acquire and maintain (approximately \50k USD plus ongoing costs), often involve long waiting lists, and have relatively short life expectancies. While robot guide dogs offer a promising alternative, existing approaches exploring this idea suffer from several drawbacks: They often lack the autonomy required for real‑world deployment, relying on prior 3D scans of the environment, external computation, or limited awareness of the handler. In this work, we present Milo, the first open‑source, low‑cost (approximately \2k USD) robotic guide dog platform capable of fulfilling the basic collaborative navigation role expected of a guide dog. Milo is fully autonomous, requiring no a priori knowledge of the environment, completely self‑contained with all computation performed onboard, and suitable for both indoor and outdoor navigation while avoiding obstacles and pedestrians. Our system consists of a modified Unitree Go2 robot (equipped with onboard compute, sensors, and a handle), a perception stack combining voxel mapping with floor, obstacle, and pedestrian detection, and a navigation stack based on an obstacle‑avoidance policy trained in a custom bird's‑eye‑view simulator. We evaluate Milo in real indoor and outdoor obstacle courses and compare it against a costmap‑based baseline, demonstrating smoother navigation and fewer handler collisions. To maximize accessibility for BLV users, we release both the robot hardware instructions and the complete software stack as open source.
Authors:Eric Price, Kevin Tian, Zhiyang Xun, Yusong Zhu
Abstract:
Modern LLM deployments use a number of implementation choices and inference optimizations (e.g., batching, custom kernels, and quantization) on top of fixed weights, so two engines serving "the same model" can produce meaningfully different distributions. We study the problem of estimating the total variation (TV) distance between two length‑n autoregressive distributions to additive error \varepsilon, under three access models. (1) Under sample access, we use \widetildeO(n^2 K/\varepsilon^2) queries, where K is the maximum support of the next‑token distribution. This improves upon the \widetildeO(n^3 m/\varepsilon^5)‑query estimator of Meel et al. (2025), where m \geq K is the total size of the token alphabet. (2) Under logit access, we use O(n/\varepsilon^2) queries, and this is tight. (3) Under noisy logit access, we smoothly interpolate between the above two guarantees: if probability values are given to relative error σ, we use \widetildeO((n+n^2σ^2)/\varepsilon^2) queries. We complement our theoretical results with an empirical evaluation of our algorithms, for example measuring the distance between SGLang and vLLM serving identical weights. Our experiments highlight the robustness and practicality of estimating the total variation distance, which remains estimable where the KL divergence is infinite. Our code is available at https://github.com/XunZhiyang/llm‑tv‑estimation.
Authors:Mert Onur Cakiroglu, Mehmet Dalkilic, Hasan Kurban
Abstract:
Detectors for AI‑generated video are evaluated offline. A clip is decoded to pixels and scored once, increasingly by a large vision‑language model. Detection, however, is deployed online. We recast the task as streaming perception and score the motion field the codec already wrote into the bitstream. Reading that field is a parse, not a pixel‑domain forward pass. Because the running aggregate is monotone, one end‑calibrated threshold is anytime‑valid at the data‑dependent decision time. Recalibrating at each prefix is not. Escalation is priced in closed form. A compute budget maps to a deferral window, on a frontier monotone exactly where the deferral condition holds. On matched GenVidBench the codec stage reaches full‑length AUC 0.64 at five orders of magnitude less compute than a pixel CNN, on CPU. Its gate holds the stopping‑time false‑positive rate at target while the real data match its calibration, and drifts above it under distribution shift. Deferring 15% of clips lifts accuracy from 0.75 to 0.78 at 7× less compute (paired: McNemar p<10^‑6). The stage‑1 ordering replicates on AIGVDBench. We introduce no new detector. The contribution is the reframing, two guarantees, and the measured frontiers. Code, configurations, and evaluation splits: https://github.com/KurbanIntelligenceLab/streamdet.
Authors:Ayoub Jadouli
Abstract:
We audit whether candle‑based machine‑learning models can turn predictions of cryptocurrency extrema or short‑horizon outcomes into positive Binance Spot paper policies after assumed costs. Numerical results come from scripted fixed‑seed model runs and deterministic simulators; human‑supervised AI agents supported the July 20 evidence‑integrity revision through literature retrieval, separately tasked critique, artifact reconciliation, documentation, and source packaging, not trading decisions. The strongest later‑period evidence, conditional on extensive predecessor search, is negative: an unchanged ten‑pair mandatory‑daily selector lost 6.72% over 19 July cycles at an assumed 31‑bps completed‑cycle cost, with 3 wins and 16 losses. In short model‑specific July evaluations, the validation‑selected local‑minimum policy returned ‑1.79%, while the local‑maximum sell‑to‑cash/re‑entry policy underperformed continuous holding by 2.80%; their gross mean advantages of 11.11 and 12.21 bps were below even the 21‑bps stress. A Gurgul‑inspired, OHLCV‑only daily adaptation attained minimum/maximum ROC AUC of 0.874/0.896 but average precision of only 0.134/0.116 and lost 44.30% over seven cycles, versus ‑41.20% for buy‑and‑hold. A forensic audit also downgraded an earlier One4All "30‑day holdout": its dates had influenced prior architecture work, its four‑hour outcome horizon was not purged at split boundaries, it used same‑close entry, and its raw result directories were absent. Across the tested, mostly exploratory protocols, event‑ranking performance did not establish positive executable policy value. Every operational decision remains NO\_TRADE.
Authors:Fabian Waschkowski, Prabod Rathnayaka, Lukas Wesemann
Abstract:
Apple's M5 generation introduces a redesigned GPU architecture in which every core carries a dedicated Neural Accelerator: on‑die matrix units exposed through the Metal~4 tensor API. We show that BaseRT, our native Metal inference runtime for large language models on Apple Silicon, exploits these units to push inference throughput on Apple hardware substantially beyond both llama.cpp and MLX. Building on BaseRT's framework‑free design, we add a family of hand‑written Metal~4 tensor‑core kernels (including dense and mixture‑of‑experts GEMM and flash‑attention prefill kernels) that route the compute‑bound matrix multiplications of inference through the M5 Neural Accelerators while leaving the memory‑bound decode path on our existing specialised kernels. On an Apple M5 Pro, across fifteen model configurations spanning the Qwen3, Qwen3.5/3.6, Llama~3.2, and Gemma~4 families from sub‑1B to 35B parameters, BaseRT delivers up to 6.4× higher prompt‑processing throughput than llama.cpp and 3.9× higher than MLX, with the largest margins on the mixture‑of‑experts models where matrix multiplication dominates, while maintaining its lead on decode of up to 1.75× over llama.cpp and 1.33× over MLX. These results establish a new performance ceiling for on‑device LLM inference and show that the M5's tensor cores are the decisive lever for prompt processing on Apple Silicon. BaseRT is publicly available at https://github.com/basecompute/baseRT.
Authors:Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li, Weijuan Zhang, Yankai Rong, Junyi Yao, Shengzhi Zhang, Xiaoqi Jia
Abstract:
The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates. We propose JailMeter, an evidence‑based evaluation framework designed to more faithfully measure jailbreak effectiveness. Inspired by the Information Bottleneck theory, JailMeter applies dual‑feedback optimization to filter jailbreak noise from model responses while preserving content relevant to the original malicious question. This process produces concise evidence for a rigorous assessment under which an attack is validated only when the response captures the malicious intent and delivers a complete answer, thereby signaling a substantive bypass of model safety alignment. We evaluate JailMeter on JailMeter‑Eva, a challenging benchmark containing 330 human‑labeled, non‑rejected jailbreak instances. JailMeter achieves an accuracy of 97.27%, substantially outperforming existing evaluation methods. To support large‑scale evaluation, we further distill JailMeter into a small language model, JailMeter\textsubscriptSLM, which maintains comparable reliability with significantly reduced computational costs. Code and dataset are available at https://github.com/Magi2B0y/JailMeter.
Authors:Dmitriy Poyarkov, Aleksei Staroverov, Aleksandr I. Panov
Abstract:
It is commonly observed that online reinforcement learning (RL) produces better‑performing strategies than offline methods across a broad range of performance measures. In particular, RL‑trained policies exhibit stronger out‑of‑distribution (OOD) behavior, where models trained only with imitation learning approaches often struggle. A recent study introduced an OOD‑focused benchmark and reported that RL‑trained vision‑language‑action (VLA) policies achieve noticeably better OOD performance and slightly better in‑distribution (IND) performance than their counterparts trained with supervised fine‑tuning (SFT). In this work, we investigate whether hybrid offline‑online training can combine the advantages of both approaches. Specifically, we study RL methods regularized by offline supervision via either offline data or an offline‑trained reference policy. We evaluate these approaches on the OOD benchmark and compare them with both offline‑only training and standard RL. Our results show that although offline training achieves limited OOD performance by itself, incorporating offline supervision into RL preserves strong OOD capability while substantially improving training efficiency. In particular, the guided methods reach performance close to that of standard RL while requiring roughly half of the training budget. Rather than producing a trade‑off between speed and OOD performance, the hybrid approach retains strong OOD capability while achieving this efficiency gain. Project page: https://alstar8.github.io/offline‑supervision‑vla‑rl
Authors:Junyi Wang
Abstract:
Multi‑entity compositional questions pose significant challenges to existing retrieval‑augmented language models. Conventional methods fall into a dilemma: standard RAG lacks dynamic reasoning, traditional Graph‑RAG is limited by structural sparsity, and LLM‑constructed Graph‑RAG incurs prohibitive costs. We propose \fwa, a unified framework that embeds unstructured text into structured knowledge graphs, creating a heterogeneous network for flexible evidence retrieval. Reasoning is formulated as adaptive structure induction, learned via a robust two‑stage process: (1) imitation learning distills heuristic expert signals, and (2) reinforcement learning refines the policy using LLM‑driven preference rewards. Experiments demonstrate that \fwa effectively merges textual richness with structural knowledge, outperforming SOTA baselines in answer accuracy and reasoning fidelity while maintaining extremely low token costs and near real‑time inference((code available at https://github.com/wjywjy123/HyGRL) .
Authors:Yihan Wang, Zhong Guan, Haoran Sun, Jiale Huang, Likang Wu, Hongke Zhao
Abstract:
Small language models are attractive backbones for interactive agents, but direct distillation from strong teacher trajectories often turns rich multi‑turn behavior into one‑shot imitation targets. This is inefficient in long‑horizon environments, where early decisions shape later states and rewards. We propose Prefix‑GRPO, a reinforcement learning framework that decomposes teacher trajectories into replay‑aligned prefix queries and online continuations. Each prefix is replayed in the environment to recover a valid intermediate state, after which the student continues online interaction and receives task reward. Unlike response‑only GRPO, Prefix‑GRPO also applies clipped policy updates to historical assistant tokens inside the replayed prefix, using a policy‑distilled SFT checkpoint to estimate their old log‑probabilities. This unifies prefix learning and continuation learning within the same policy‑optimization form. Experiments on TextCraft, BabyAI, and ALFWorld show that Prefix‑GRPO improves small‑model agents over distillation and standard RL baselines, while ablations show that replay alone is insufficient without explicit prefix‑token optimization. The implementation and reproduction scripts are available at https://github.com/HappynessI/Prefix_GRPO.
Authors:Keston Aquino-Michaels
Abstract:
A recent report finds that orthogonalizing the mLSTM memory matrix at read time (five Newton‑Schulz iterations, trained through) substantially improves noisy associative recall. The effect replicates, but it is not a memory improvement. Training on this task is a long chance plateau followed by a sharp escape, and the orthogonalized read acts by re‑conditioning the learning problem during the plateau. Three properties establish this. It must be self‑consistent: an exact recursive least‑squares read (the Mesa layer) reproduces it, while straight‑through halves, delta‑rule writes, frozen random keys, and plain normalization all fail. It is uniform: across a learning‑rate x hardness grid it multiplies the escape hazard roughly six‑fold with no detectable hardness dependence, widening the workable learning‑rate corridor that narrows for the baseline. And it is removable: applied to failed models at inference it rescues none, and annealed away on an escape‑triggered schedule it leaves numerically stock mLSTMs at full accuracy. Much of the published gain needs no architecture at all: solved‑rate at a fixed budget measures escape hazard, which follows a heat/noise law (learning‑rate elasticity +3.0, gradient‑noise elasticity ‑1.65) under which the original vocab‑96 result is a large‑batch noise condition rather than a capacity one. Decoding the memory state directly shows failed models carry roughly half their associations in linearly recoverable form: the plateau is a readout failure over half‑written storage. Two conclusions travel beyond the intervention: recall benchmarks used for architecture selection partly measure trainability, and the system is a fully instrumented model organism of "emergence," in which a sharp behavioral threshold demonstrably arises from a censored metric over gradually accumulating structure.
Authors:Linbin Tang, Jingyan You, Zilin Kang, Hanzhang Liu, Sophia Zhang, Zenan Li, Chenrui Cao, Liangcheng Song, Jiaao Wu, Xian Zhang, Fan Yang
Abstract:
Recent formal reasoning systems have reached IMO‑level performance, yet they leave a fragmented landscape: algebra and number theory are handled in Lean, while geometry still relies on domain‑specific languages with limited formal guarantees. This split increases the trusted computing base and hinders unified model development. Existing geometry‑in‑Lean efforts (LeanEuclid, LeanGeo) introduce custom axiom systems incompatible with standard Mathlib, and their small scale (< 1,100 problems) limits large‑scale training. Native Mathlib autoformalization of geometry, however, poses distinct challenges: implicit diagrammatic assumptions (e.g., topological configuration and non‑degeneracy) must be made explicit rather than deferred to external solvers, and models must adapt to Mathlib's small, rapidly evolving geometry infrastructure. We present Euclean, a four‑stage framework ‑ constraint explication, configuration anchoring, formalization mapping, and iterative repair ‑ for automatically formalizing geometry in native Mathlib. We construct OMNI‑Geometry (768 competition problems) and Numina‑Geometry (177,597 problems), the largest geometry formalization dataset in Lean. Human evaluation shows 48.89% TOP1 and 73.33% TOP5 accuracy. Training Goedel v2 on our formalizations improves proof success from 13.6% to 15.1%, validating dataset quality for unified neural theorem proving. Code and datasets: https://github.com/tlb‑22/Euclean.
Authors:Oshayer Siddique, J. M Areeb Uzair Alam, Md Jobayer Rahman Rafy, Syed Rifat Raiyan, Hasan Mahmud, Md Kamrul Hasan
Abstract:
Activation steering offers a lightweight alternative to fine‑tuning for behavioral control of large language models, but SAE‑based steering methods often rely on learned steering objectives or single‑criterion feature selection. We introduce a transparent SAE‑feature steering pipeline that first applies a six‑condition reliability filter, then ranks sparse features through an unweighted Borda consensus over three complementary statistics: F‑test, KSG mutual information, and Cohen's d. The resulting steering direction is constructed as a Cohen's‑d‑weighted combination of SAE decoder rows, providing an optimization‑free direction motivated by Fisher‑LDA under approximate SAE‑feature decorrelation. Across three Gemma‑family models, four behavioral domains, and 356 layer‑strength configurations, the method produces measurable domain‑specific shifts while revealing a substantial gap between raw attribute movement and quality‑preserving generation. In the strongest configuration, logical‑correctness steering reaches a primary‑score delta of +1.16 in Gemma~2 9B; however, our broader finding is that usable steering is highly localized by model, domain, layer, and strength. These results argue that activation‑steering evaluations should report quality‑conditioned success alongside raw behavioral shift. Our code and data are available at https://github.com/Oshayer‑Siddique/LLM‑Steering‑Using‑SAE.
Authors:Shengtong Zhu
Abstract:
Long‑term memory is essential for LLM agents that interact across sessions, yet current memory benchmarks primarily evaluate single‑hop recall, leaving multi‑hop association largely unmeasured. We make three contributions. First, we introduce MemHop, a multi‑hop memory benchmark of 1,000 questions at hop depths 1‑5 across 10 social‑network scenarios, with per‑hop evidence annotations. Second, we present Profile‑Graph Memory (ProGraph), a two‑layer memory architecture combining (i) profile expansion ‑‑ substring‑matched traversal of entity names that naturally appear in LLM‑written profile narratives, a minimal alternative to explicit knowledge‑graph construction ‑‑ and (ii) compression residuals ‑‑ exact dates, quantities, and named items co‑extracted with each profile update at zero extra API cost. Third, a full‑grid ablation shows cross‑benchmark mechanism specialization: profile expansion drives multi‑hop reasoning (‑22.6pp on MemHop when removed) while compression residuals drive precision recall (‑8.6pp on LoCoMo when not co‑extracted), with cross‑effects under 3pp within a single architecture. ProGraph averages 80.1% on MemHop (matching the FullContext reference) and 78.4% on LoCoMo (exceeding FullContext by 11.3pp), outperforming Mem0, A‑Mem, HippoRAG, and RAG on both. We release MemHop, ProGraph, and baseline implementations.
Authors:Tiancheng Zhang, Shaoyuan Huang, Mingyuan Wang, Yunfeng Zhao, Xiaofei Wang, Wenyu Wang
Abstract:
Large language models (LLMs) are increasingly deployed as always‑on online services, making efficient LLM serving a critical systems challenge. Achieving low latency and high throughput under volatile demand requires deep understanding of real‑world serving workloads, yet existing studies often rely on proxy traces or coarse‑grained characterizations that fail to capture the heterogeneity of modern multi‑model LLM platforms. We present FineServe, an in‑the‑wild, multi‑model LLM serving workload dataset collected from a global commercial marketplace, enabling fine‑grained characterization of real‑world serving dynamics across heterogeneous models and tasks. Leveraging FineServe, we conduct a comprehensive analysis of arrival dynamics and token behavior, revealing fundamentally different fluctuation regimes across model architectures, scales and task intents. Building on these insights, we develop the FineServe workload generator, which composes fine‑grained model‑aware workloads into configurable mixtures tailored for benchmarking multi‑model serving platforms. By exposing these fine‑grained workload dynamics, FineServe provides a realistic foundation for evaluating routing, scheduling, and capacity‑planning strategies in LLM serving systems. FineServe is available at https://github.com/hihiztc1/FineServe.
Authors:Yu Chen, Caorui Li, Ziyu Xiong, Yidong Wang, Mingqi Gao, Shuman Liu, Biao Liu, Chunfeng Yang, Anxiang Zeng, Haibo Zhang, Chaofan Chen
Abstract:
Long audio‑video reasoning is difficult for omnimodal LLMs because the decisive evidence is often sparse, cross‑modal, and too expensive to preserve with uniformly high‑fidelity inputs. We introduce OmniReasoner, a tool‑use post‑training framework for Thinking with Long Audio‑Video: omni‑modal LLMs learn, via supervised fine‑tuning and reinforcement learning, to decide whether and where to call a zoom‑in tool before answering. OmniReasoner first builds a low‑cost global preview of the full stream and then, when needed, calls the zoom‑in tool with a requested temporal interval for higher‑fidelity visual and audio inspection before answering. Because the model observes different sampling granularities before and after this call ‑‑ a sparse global preview and a denser local clip ‑‑ we introduce TimeAnchor, which keeps the tool's temporal argument valid and round‑trip‑consistent across these granularities, rather than tied to frame indices from a particular sampling rate. To make this tool‑use behavior trainable without expensive manual interval annotation, we build a Temporal Augmented Data Engine that synthesizes tool‑use post‑training trajectories by video editing and composition. Experiments across omnimodal and video benchmarks show that OmniReasoner improves both answer accuracy and temporal grounding while concentrating high‑fidelity computation on informative regions. Code is available at https://github.com/RockyChen0205/OmniReasoner.
Authors:Qijia He, Jiayi Cheng, Chenqian Le, Rui Wang, Xunmei Liu, Yixian Chen, Jie Mei, Zhihao Wang, Xupeng Chen, Yuhuan Chen, Tao Wang
Abstract:
Coding agents increasingly operate in executable environments where a failed attempt produces actionable feedback rather than merely an incorrect answer. Existing cost‑aware systems typically treat such failures as cascade decisions: try a cheap model first, then escalate hard cases to a stronger and more expensive model. In coding, however, execution feedback can also make further cheap‑model recovery worthwhile, raising a budgeted deployment question: when should an agent spend more cheap compute, and when should it escalate? We formulate this post‑failure decision as recovery routing over heterogeneous actions and train a supervised router from execution rollouts. To make the same router usable under changing budgets, we add a Conformal Risk Control (CRC) layer that selects a deployment‑time cost penalty without retraining and provides marginal expected‑cost control under exchangeability. Across held‑out failures from five coding benchmarks, cheap recovery and escalation exhibit complementary success patterns. The calibrated frontier improves over fixed actions, prompt‑only routers, and a binary cascade baseline; in the main GPT‑5.4‑nano/GPT‑5.4 setting, one CRC‑calibrated frontier point exceeds always‑escalate solve rate while using 35% of its mean recovery cost. Code is available at https://github.com/Qijia‑He/agent‑budget‑control.
Authors:Pratinav Seth, Hem Gosalia, Aditya Kasliwal, Vinay Kumar Sankarapu
Abstract:
Circuit analysis can support not only model explanation but also downstream interventions such as pruning, editing, steering, and selective fine‑tuning. However, conducting such analyses currently requires stitching together separate implementations for discovery, evaluation, and intervention, as well as hand‑authoring the contrastive prompts required by many discovery methods. This fragmentation makes methods difficult to compare and limits their application beyond canonical tasks. We introduce CircuitKIT, a source‑available library that connects the circuit‑analysis workflow through a typed, serializable representation. CircuitKIT provides a suite of discovery algorithms, declarative interfaces for mapping structured data into discovery tasks, complementary circuit diagnostics, and downstream application modules. Together, these components provide common infrastructure for conducting and comparing circuit analyses. The library, examples, notebooks, and documentation are released at https://github.com/Lexsi‑Labs/CircuitKIT .
Authors:Xingyu Chen, Junxiu An, Jun Guo, Li Wang
Abstract:
GraphRAG improves long‑document question answering by introducing structured representations beyond conventional retrieval. However, automatically constructed graphs are inherently incomplete projections of source documents, and treating them as independent knowledge sources may lead to unreliable retrieval and generation. We propose PAGE‑RAG, a projection‑aware adaptive graph retrieval framework for reliable long‑document question answering. PAGE‑RAG views graph structures as semantic skeletons that organize and navigate document knowledge, rather than replacing the original knowledge source. Based on this perspective, PAGE‑RAG introduces a task‑adaptive retrieval routing strategy that dynamically selects appropriate retrieval behaviors according to query requirements. Furthermore, PAGE‑RAG incorporates strict knowledge boundary control, ensuring that generated responses remain grounded within available evidence and abstaining from unsupported information beyond the accessible knowledge scope. Experiments demonstrate that PAGE‑RAG achieves competitive answer quality while improving retrieval efficiency and knowledge reliability, highlighting the importance of projection‑aware graph modeling, adaptive retrieval, and explicit knowledge boundary control for trustworthy GraphRAG systems. The source code is publicly available at https://github.com/CXY0112/PAGE‑RAG.
Authors:Yohann Sidot
Abstract:
We study a five‑agent CI/CD pipeline (triage ‑> developer ‑> security‑scan ‑> review ‑> approve/deploy), built from five distinct production LLMs across three providers, behind an LLM firewall in shadow mode. A single untrusted input ‑ an external issue requesting a "usage‑telemetry" feature ‑ asks for code that exfiltrates process secrets (dict(os.environ)) to an attacker URL, laundered as observability. Across a pre‑registered A x B (x C) factorial (N=20; naive arm N=60) we find: (1) the entry agent does not leak its system prompt (0/40); (2) an authority‑framed injection ("pre‑approved under SEC‑2291, do not re‑review") makes downstream verifiers see the secret‑exfil line, cite the pre‑approval, and ship it ‑ the scanner passes ~80% of laundered pull requests, and the worst‑case cell reaches 55% compromise; (3) the perceived presence of other verifiers yields only a small, non‑significant reduction in individual scrutiny (a weak bystander analogue), even at N=60; and (4) content‑based controls ‑ code scanners and pattern detectors alike ‑ miss the laundered intent entirely (the code is syntactically clean); only an LLM reasoning about intent is a partial defence. The failure is systemic: neither prompt secrecy nor distributed verification protects; a provenance‑aware control at the entry, independent of both, would have. All data is 100% synthetic; the sink is mocked and the exfil URL is never contacted.
Authors:Rahil Sharma
Abstract:
Fraud detection systems must scale with rising transaction volume while remaining explainable and reviewable. We study a layered pipeline on the PaySim dataset that combines a gradient‑boosted classifier, graph‑derived structural features, an autoencoder‑based anomaly signal, TreeSHAP explanations, and a bounded LLM investigation agent applied to cases the classifier scores uncertainly. Before any model comparison, we identify and remove a simulator‑specific balance shortcut that would otherwise inflate baseline performance. After this correction, neither the graph features nor the anomaly signal improves Average Precision on the full test set. Both, however, rank fraud better within the subset of cases receiving intermediate baseline scores. In a controlled experiment with injected multi‑account fraud rings, engineered structural features recover all injected test transactions, while the tabular baseline misses roughly a quarter of them. The investigation agent underperforms direct thresholding of the classifier it relies on, reaching 65.0% accuracy against 71.7% on a balanced 60‑case sample, despite having access to model explanations, graph context, and retrieved reference cases. Of the eight decisions the agent changed, six replaced correct classifier outputs with errors, and it produced a coherent written rationale in each case. An exploratory disagreement‑based escalation rule flagged two of these agent errors for human review without flagging any correct decision. We conclude that each component of a layered fraud system contributes only under specific conditions, and that a plausible rationale from an investigation agent is not evidence of a better decision.
Authors:Netanel Eliav
Abstract:
Practitioners make three prompt‑design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade. We report two controlled experiments crossing all three factors on one held, contamination‑free synthetic corpus (the "Book of Veyra," 8,780 uniquely‑named entities, deterministically regenerable from a fixed seed), evaluated across five models. Experiment 1 (960 calls/model) measures instruction‑following decay as rule count N grows from 10 to 160, crossed with four formats and system‑prompt vs. user‑turn placement. Perfect‑response rate collapses to zero by N=80 for every model, format, and placement. Placement produces effects at least as large as format at N=160 in most models, but the direction is model‑specific. No model shows a reliable markdown advantage; one 35B model favors plain text instead. Experiment 2 (5,520 calls/model) measures recall accuracy, false‑premise sycophancy, and absent‑fact fabrication across a 2k‑to‑512k‑token context ladder in the same four formats. Recall stays near ceiling through 64‑128k tokens, then degrades sharply and format‑dependently: one model's accuracy spread reaches 48 points at 128k tokens. Fabrication never occurs (0/5,760 probes), and sycophancy stays negligible (<=8.3%). What rises sharply near each model's context ceiling is outright refusal to answer (0% to 79‑90%), distinct from sycophancy or fabrication. Neither pre‑registered format ordering holds, and token overhead (+22% to +37% over plain text) further changes which format is preferable where accuracy spread is genuine. We release the full harness, corpus generator, and raw results (VeyraBench): https://github.com/iNetanel/veyrabench
Authors:Haram Choi
Abstract:
Prior work on human label variation (HLV) in natural language inference (NLI) has often relied on re‑annotation resources that select items by disagreement level. An earlier study (arXiv:2607.15870) found that hypotheses containing non‑upward monotonicity operators showed lower label agreement in ChaosNLI (Cliff's delta = ‑0.284), which is restricted to items whose majority label carries exactly three of five votes. We preregistered a replication of this boundary in the unselected populations that ChaosNLI was drawn from: the SNLI and MultiNLI development sets, using the same operator tagger and a four‑level ordinal agreement outcome. The registered prediction fails. All seven contrasts return a positive Cliff's delta (non‑upward items agree slightly more, not less), the only significant confirmatory contrast has the opposite sign to the registration, and every effect is far below our smallest effect size of interest (0.10). Robustness checks support the measurement: simulated tagger misclassification shrinks the effects rather than manufacturing them, and a manual re‑tagging audit reaches four‑class agreement of 0.875 on a fresh 200‑item sample. We conclude that the earlier negative boundary is plausibly a structure conditional on low‑agreement selection rather than a population‑level property, and that HLV structure claims built on selected re‑annotation resources should state their selection conditional explicitly.
Authors:Xuefeng Jin, Jiashuo Zhang, Teng Cao, Bin Yang
Abstract:
Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG). However, existing studies rely primarily on prompt engineering or supervised fine‑tuning, while systematic research on reinforcement learning (RL) post‑training and automated evaluation of feedback quality remains limited. We propose RLAES, a unified LLM framework that jointly optimizes essay scoring and feedback generation through RL. To make feedback quality measurable, interpretable, and usable for training, we introduce Rubric‑based Feedback Evaluation (RFE), an essay‑grounded feedback evaluation framework comprising 166 fine‑grained binary rubric items and an LLM‑as‑judge. Building on RFE, we propose Adaptive Gated Feedback Optimization (AGFO), which activates rubric‑based feedback rewards on demand during RL, reducing evaluation overhead while improving feedback quality. We also propose Adjacent Contrastive Reasoning (ACR) to improve ordinal score calibration by explicitly contrasting adjacent score levels. Experimental results show that the RFE framework captures essay‑feedback consistency, exhibits strong pairwise discriminative power, and closely aligns with expert preferences. On the ASAP benchmark, RLAES‑AGFO achieves the best scoring performance among LLM‑based methods (QWK = 0.803), while maintaining feedback quality comparable to GPT‑5.5 and avoiding the feedback degradation observed under score‑only RL. Code and datasets are publicly available at https://github.com/hellomuyi/RLAES.
Authors:Maxim Khailo
Abstract:
Frontier LLM providers cache a prompt's processed prefix so that a follow‑up request sharing it pays ~10% of the input price and skips most of the prefill latency. Agentic workloads systematically destroy this benefit: the agent sends a request, runs a tool or waits for approval for minutes, and by the time the follow‑up is sent the cached prefix has been evicted, so the agent pays the full prefill again. A client‑side keepalive, replaying the prefix on a timer during the pause, prevents this, and it is individually rational: across Anthropic, OpenAI, Google, and DeepSeek we show that a keepalive holds the prefix warm through gaps where idle baselines are evicted, cutting the post‑pause request cost by up to 12.5x. The strategic question is the ping frequency, and it has a clean answer: keepalive cost falls monotonically in the interval, so the economical choice is the largest interval safely under the provider's TTL, about 4 minutes at Anthropic's 5‑minute TTL rather than the 30‑second convention, and the strategy breaks even against a re‑prefill at idle ~tau(w/r ‑ 1) (~46 min for Anthropic, ~36 min for OpenAI and DeepSeek). Because the benefit is real and bounded only by each user's own bill, rational adoption is universal adoption; and since cache residency is priced per read rather than per token‑hour, a keepalive‑saturated tier gives LRU eviction nothing to rank. We argue this externality will push providers to meter cache residency directly, and one already does. We derive the operator's policy until then.
Authors:Guanxiong Chen, Qianjun Xia, Jiawei Peng, Heng Zhang, Bole Ma, Justin Qian, Ziyi Jiao, Bingyang Zhou, Luoxin Ye, Kaifeng Zhang, Kunyi Wang, Weijia Zeng, Yunuo Chen, Pengzhi Yang, Ziqiu Zeng, Siyuan Luo, Huamin Wang, Chao Liu, Alan Yuille, Fan Shi, Changxi Zheng, Yunzhu Li, Chenfanfu Jiang, Peter Yichen Chen
Abstract:
Real‑to‑sim conversion for robotic interaction with objects remains labor‑intensive because it requires more than visual reconstruction: a streamlined real2sim process must recover scene geometries and object states, infer physical parameters, and assemble actors, objects, cameras, poses, and trajectories into a runnable physical simulation. Today this process still depends on manual tuning of visual foundation models, mesh cleanup, coordinate‑frame alignment, and brittle workflow glue across visual perception tools and simulators. We introduce Agentic Real2Sim, a framework for generalized physical world modeling with vision‑language agents, converting a real‑world recording of object‑robot interaction into a simulatable episodic twin which preserves observations, geometries, robot interactions, and object states. We evaluate Agentic Real2Sim on rigid‑object manipulation, deformable‑object interaction, and humanoid motion scenes, spanning domains that are usually handled by separate Real2Sim pipelines, marking a first step toward scalable conversion. The framework's agentic decisions can be driven by an open‑weight VLM backend at a small fraction of the cost of frontier models, while attaining comparable conversion success rate. We aim to use the resulting real‑world‑aligned twins for downstream robotics tasks, specifically policy learning and evaluation. The project site is available at https://agentic‑real2sim.github.io/.
Authors:Aixiu An, Michael Jungo, Eloi Eynard, Mark Drenhaus, Andreas Fischer, Jean Hennebert, Sébastien Rumley
Abstract:
Neural machine translation (NMT) in the legal domain is a linguistically and conceptually demanding task, primarily due to the complexity of legal language and the high level of precision it requires. The recent emergence of reasoning‑capable language models opens new possibilities for tackling such challenges. They add to a set of other previously proposed techniques to enhance the translation quality, which includes supervised fine‑tuning and reinforcement learning. In this work, we perform a comparison between these various approaches. More particularly, we evaluate small language models such as Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B enhanced with various re‑training paradigms and compare their performances against frontier reasoning models. We focus on the Swiss legal system, which ‑‑ with its unique multilingual statutes ‑‑ offers a particularly challenging testbed for reasoning‑augmented models. Our results show that the quality of small ``base'' models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine‑tuning. The performance of enhanced small models is close to the one of state‑of‑the‑art reasoning models yet remains inferior. We also note that re‑training paradigms yield diminishing returns as model size increase. The code and models are publicly available at https://github.com/aixiuxiuxiu/Legal‑MT‑SFT‑RL.
Authors:Junlin Chang, Longhao Zou, Rui Li
Abstract:
Fine‑tuning pre‑trained point‑cloud backbones typically updates all parameters, resulting in substantial computation and memory overhead. More importantly, modern point backbones rely on aggressive tokenization and downsampling, which yields compact global tokens but irreversibly discards fine‑grained local geometry, an inherent bottleneck for parameter‑efficient adaptation. Consequently, existing PEFT methods that operate only on these coarsened tokens can modulate global semantics but struggle to recover the missing multi‑scale locality. We present Point Ladder Tuning (PLT), a locality‑aware PEFT framework that performs hierarchical, instance‑conditioned adaptation while keeping the backbone frozen. PLT forms a lightweight closed loop: (i) a Hierarchical Ladder Network (HLN) constructs a multi‑resolution local feature pyramid directly from raw points; (ii) a Local‑Global Fusion (LGF) aligns and fuses local pyramids with intermediate backbone semantics; and (iii) a Dynamic Prompt Generator produces instance‑aware multi‑scale prompts to modulate the frozen backbone effectively. For dense prediction, we further introduce a lightweight segmentation head that progressively upsamples fused features and leverages backbone priors to refine fine structures. Extensive experiments on classification and dense prediction show that PLT consistently surpasses prior PEFT baselines with minimal tunable parameters. PLT achieves state‑of‑the‑art performance using only 2.71% trainable parameters for classification and 7.69% for dense prediction, and scales favorably to larger backbones, requiring merely 0.36% parameters on PointGPT‑L. The code is released at https://github.com/JunLinChang/ECCV2026‑PLT.
Authors:Jiayi Yang, Yifang Chen, Yuanfu Sun, Jiajin Liu, Qiaoyu Tan
Abstract:
Vision‑language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general‑purpose backbones for graph‑structured data remains largely unexplored. In practice, attributed graphs exhibit substantial modality heterogeneity: some graphs contain only textual node attributes, others only visual attributes, while still others provide both. Existing graph learning approaches are typically designed for fixed modality schemas, requiring separate models for different settings and limiting scalability and cross‑graph generalization. To bridge this gap, we present OMG‑VLM (One Model, Many Graphs with Vision‑Language Models), a unified framework for learning over attributed graphs across heterogeneous modality schemas. OMG‑VLM leverages a pretrained VLM as a shared backbone and introduces structure‑aware graph adapters that integrate neighborhood information while remaining compatible with the VLM's native embedding space. This design enables effective learning over text‑attributed, image‑attributed, and multi‑attributed graphs within a single model. Extensive experiments across diverse domains show that OMG‑VLM consistently outperforms state‑of‑the‑art GNN‑ and LLM‑based baselines on attributed graph learning tasks such as node classification and link prediction, while exhibiting strong generalization to unseen graphs and varying modality schemas. The source code is available at https://github.com/Jo‑eyang/OMG‑VLM.
Authors:Lisa Weijler, Irene Ballester, Guofeng Mei, Tolga Birdal, Pedro Hermosilla
Abstract:
Geometric foundation models, such as the Visual Geometry Grounded Transformer (VGGT), provide strong 3D priors from unposed images. However, such models operate purely in a feed‑forward, deterministic regime, \ie~they cannot generate plausible geometry beyond what the input views directly support. Generative models for 3D scenes, on the other hand, must rely on strong geometric priors to produce coherent outputs from sparse inputs. We bridge these two paradigms by performing flow matching directly in VGGT's latent space, leveraging its learned 3D priors without committing to any explicit downstream representation such as Gaussians, meshes, or video‑VAE latents. This requires respecting the latent geometry: VGGT tokens occupy a product of high‑dimensional hyperspheres on which standard Euclidean flow matching fails. We address this with a Riemannian Flow Matching framework defined on a product manifold of four hyperspheres, aligned with VGGT's multi‑scale encoder, which keeps generated tokens on the valid data manifold required by the frozen decoding heads. On RealEstate10K, ScanNet++ and ETH3D, our method achieves strong performance against recent scene generation baselines in both per‑view appearance and aggregated 3D geometry, establishing latent‑space flow matching on geometric foundation models as a viable paradigm for 3D generation. The project page can be found \hrefhttps://lisaweijler.github.io/geometry‑grounded‑rfm/\texthere.
Authors:Shimon Murai, Fangzheng Lin, Kasidis Arunruangsirilert, Jiro Katto
Abstract:
Autoregressive context models are foundational for learned image compression,but they suffer from slow serial inference. Existing acceleration methods such as checkerboard context require architectural changes and retraining, thus are inapplicable to pre‑trained models. We propose a completely training‑free inference‑time acceleration algorithm inspired by wavefront parallelism in video coding standards. Our method reorganizes inference into an optimal ``staggered'' wavefront order, minimizing sequential steps while maintaining exact autoregressive dependencies. Experimental results show our approach accelerates pre‑trained autoregressive models (e.g., Cheng et al.) by more than 13× while preserving the original rate‑distortion performance. We also demonstrate that faster decoding is possible by trading off precise context dependencies. Source code will be available at https://github.com/tokkiwa/compressai‑wavefront.
Authors:Mykola Lavreniuk, Nataliia Kussul, Andrii Shelestov, Yevhenii Salii, Volodymyr Kuzin, Charlotte Julia Li-Xing Wang, Zoltan Szantoi
Abstract:
Accurate agricultural field boundary delineation at large scale is a foundational task for food security, supply chain transparency, and carbon accounting. While vision foundation models like SAM show remarkable zero‑shot capabilities, they frequently fail in geospatial domains due to topological complexity, cropland texturing patterns, and a lack of physical scale awareness. In this work, we introduce Delineate Anything v2, a globally scalable foundation model designed specifically for wide‑area field boundary mapping. We construct FBIS‑73M, a 73‑million‑instance multi‑resolution dataset spanning 61 countries. To address the pervasive issue of multi‑field administrative parcel merging, we introduce a resolution‑specific data curation pipeline that leverages topological image‑space adaptation to homogenize merged parcels and strengthen weak physical boundaries. Furthermore, we establish a novel, manually curated evaluation benchmark covering 100 countries to assess independent zero‑shot generalization. Our results show that Delineate Anything v2 surpasses the current state‑of‑the‑art, including the Delineate Anything framework, by 0.284 mAP@0.5 (+103.3% relative gain), while maintaining execution speeds suitable for rapid national‑ and global‑scale deployment, as demonstrated by nationwide mapping of Ukraine (603,000 km^2) in 5.4 hours on a consumer‑grade workstation. Code, pre‑trained weights, the FBIS‑73M dataset, and ready‑to‑use national‑scale vector boundary products are publicly available at https://github.com/Lavreniuk/Delineate‑Anything.
Authors:Nuemaan Malik
Abstract:
Optimizer state is the largest single line item in the memory budget of mixture‑of‑experts (MoE) training: on a 6.78B‑parameter MoE language model, AdamW keeps 50.6 GB of first and second moments to update 12.6 GB of bfloat16 weights. We study SkewAdam, an optimizer built on the observation that the three parameter populations of an MoE ‑ the dense backbone, the experts, and the router ‑ differ enough in size and gradient statistics that they should not receive the same state. SkewAdam keeps float32 momentum plus a factored second moment for the backbone (5% of parameters), a factored second moment alone for the experts (95%), and an exact second moment for the router (<0.01%). The resulting state occupies 1.29 GB, 2.6% of AdamW's, and peak training memory falls from 81.4 GB to 31.3 GB, within the budget of a 40 GB accelerator. In a controlled comparison from identical initializations over 82M tokens, SkewAdam reaches validation perplexity 108.4, ahead of AdamW (126.8), Muon (120.2), and Lion (393.7), and settles router load balance to within 1% of its uniform floor. The allocation is not what earns that perplexity: a tier ablation matches it with twenty times the state, and Adafactor, which shares the factored estimator but drops momentum, plateaus 40 points behind. The tiers buy memory at no cost to accuracy; the accuracy comes from keeping momentum, which a uniform optimizer shares too. Sweeping the baselines' learning rates narrows but does not close the gap: the best tuned AdamW reaches 118.5, tuned Adafactor 139.7. Where optimizer state lives, these results suggest, matters at least as much as how much of it there is.
Authors:Nian Liu, Yuxin Yang, Shubo Lin, Sikui Zhang, Liang Li, Boyu Cai, Yizheng Wang, Weiming Hu, Jin Gao
Abstract:
Infrared small target detection (ISTD) remains challenging because tiny, low‑contrast targets are easily overwhelmed by clutter, noise, or occlusion. Conventional single‑frame and multi‑frame detectors rely on bounding‑box supervision, which specifies final target locations but offers little explicit guidance for prioritizing candidate regions or preserving weak‑target evidence before localization. Task‑driven visual search offers such guidance: top‑down goals and visual evidence jointly form a spatial priority map that ranks candidate locations. Building on this principle, we propose Gaze‑DETR, a bio‑inspired detector that learns an internal priority map before localization. First, a priority head predicts a normalized priority map from image features. Second, Residual Priority‑Guided Feature Modulation (RPFM) enhances high‑priority responses while retaining multi‑scale features. Finally, Priority‑Guided Anchor Query Injection (PAQI) converts high‑priority locations into decoder anchor queries. We train the priority head using three supervision schemes: box‑derived Gaussian maps; real‑gaze maps constructed from fixation‑density maps; and transferred pseudo‑gaze maps learned from gaze‑‑box relations in paired annotations and applied to Anti‑UAV410 training boxes. To support the latter two schemes, we construct TIR‑UAV120‑Gaze with paired detection and task‑driven eye‑tracking annotations. On TIR‑UAV120‑Gaze, Gaze‑DETR achieves 85.76 mAP_50 and 88.77 F1 with box‑derived supervision, and 86.18 mAP_50 and 89.00 F1 with real‑gaze supervision. On Anti‑UAV410, it achieves 87.06 mAP_50 and 90.90 F1 with box‑derived supervision, and 87.08 mAP_50 and 90.43 F1 with transferred pseudo‑gaze supervision. These results show that explicit spatial‑priority learning provides pre‑localization guidance complementary to bounding‑box supervision across annotation settings and costs.
Authors:Zhihao Yang, Zhiyu Xiang, Peng Xu, Tianyu Pu, Kai Wang, Eryun Liu, Dongping Zhang, Yong Ding
Abstract:
V2X collaborative object detection features overcoming the limitations of single‑vehicle systems by aggregating environmental features from multiple collaborative agents. However, existing mainstream V2X perception methods mainly focus on 2D BEV object detection. When 3D detection task is concerned, inferior results are obtained because they ignore the 3D spatial misalignment caused by differing height and attitude among the collaborators. In this paper, we propose a novel collaborative 3D object detection framework called CoGoal3D, which extracts and refines the 3D feature gradually in a two‑stage pipeline. In the first stage, a multiscale 3D‑aware global fusion module is designed to mitigate the 3D spatial misalignment. The resulting proposals are then refined in the second stage with an auxiliary task of 3D point reconstruction. An effective multi‑agent collaborative data augmentation strategy is further proposed to enrich the training data while minimizing information loss. Extensive experiments on public real‑world datasets demonstrate that our CoGoal3D achieves new state‑of‑the‑art performance, with 3D AP@0.7 improvements of 10.86%, 10.34%, and 10.18% on the DAIR‑V2X, V2V4Real, and V2X‑Real datasets, respectively. Code is available at https://github.com/Megalo‑f/CoGoal3D.
Authors:Lei Hu
Abstract:
Existing Multi‑view Anomaly Detection (MAD) methods assume that all views are completely available and model each view separately. However, in real industrial scenarios, information in the view may be missing due to faults such as occlusion, which leads to the performance degradation of existing methods due to the lack of a multi‑view consistency prior. To address this, we explored a more challenging task: Incomplete Multi‑View Anomaly Detection (IMVAD), in which some areas of each view were masked. We proposed a pipeline for automatically generating the IMVAD dataset and generated the RIMAD dataset based on the Real‑IAD dataset through this pipeline. In addition, in order to effectively utilize the information of multiple views in the absence of view information, we propose IMMoE, which consists of two key modules: (1) Multi‑View Expert Fusion (MVEF) effectively fuses multi‑view information through a multi‑view expert network and guides the reconstruction of a single view; (2) Local Anomaly Enhancement Encoder (LAEE) effectively prevents the model from overfitting the mask region by applying dropout to local features. Our method achieves state‑of‑the‑art performance on both the RIMAD and Real‑IAD datasets, especially on RIMAD, we have increased the pixel‑level and image‑level metrics by 11.8% and 2.8%, respectively. Our source code is available at https://github.com/HULEI7/IMMoE
Authors:Fatema Ferdous Tamanna, K. M. Merajul Arefin, Md. Abdul Masud
Abstract:
Background: Clinical decision support systems degrade silently as treatment protocols evolve, yet standard adaptation methods treat models as monolithic blocks, unable to distinguish stable patient physiology from shifting institutional practice. Methods: We propose an adaptive clinical intelligence architecture for ICU intervention prediction that structurally decouples physiological from treatment representations, confining parameter updates to the treatment stream upon a dual distributional and accuracy trigger. Automated audit logs record which treatment features drove each adaptation event and how their importance shifted. At inference, an attribution‑driven Temporal RAG module grounds each prediction in patient‑specific, era‑matched PubMed evidence anchored to the patient's dominant physiological features. Experiments used 84,792 MIMIC‑IV stays (2008‑2022) under strict chronological split. Results: Drift localised entirely to the treatment stream, validating the structural prior. Selective adaptation improved vasopressor and septic shock discrimination and calibration over the static source model. A fully retrained baseline yielded marginally higher aggregate discrimination but missed 26 septic shock cases the framework correctly identified, with none in the reverse direction; retrieval consistency with the pre‑adaptation source model was preserved by the framework but degraded substantially in the retrained baseline. Conclusions: Structurally constraining adaptation to drifting components while preserving stable physiological representations enables clinical AI to evolve with practice without distorting learned patient biology. This architecture offers a template for governable, interpretable deployment of adaptive models in high‑stakes clinical environments.
Authors:Jiuhe Qu, Yingping Liang, Ying Fu
Abstract:
Change detection aims to identify semantic changes between remote sensing images. However, features from models are easily disturbed by non‑semantic variations, such as illumination, shadows, and atmospheric changes, leading to false alarms and limited generalization in real‑world scenarios. In this paper, we propose SCDistill, a framework for learning semantic‑robust change detection via semantic‑invariant self‑distillation. First, to strengthen semantic consistency, we introduce a semantic‑invariant self‑distillation strategy that learns semantic robustness from perturbed yet semantically consistent data, empowering the change detector to extract disturbance‑resistant features and achieve more reliable and accurate semantic change identification. Second, to expand paired data with non‑semantic variations, we design a diffusion‑based perturbation simulation pipeline that synthesizes complex environmental changes, enabling the model to explicitly learn to distinguish semantic changes from appearance‑level fluctuations and reduce false alarms caused by non‑semantic disturbances. These components promote robustness from data and representation perspectives, leading to synergistic performance gains. Extensive experiments demonstrate that SCDistill achieves state‑of‑the‑art performance on multiple semantic change detection benchmarks and exhibits strong generalization to binary change detection and change captioning tasks. Code is accessible at https://github.com/elecreak/SCDistill.
Authors:Xule Liu, Hanlin Teng, Chao Li, Yanan Ni, Shuo Lu, Audrey Wang, Yijun Liu, Yunfei Wang, Xiaofeng Li, Xian Yi, Yuanfa Li, Kang Zhao, Jian Liang, Yuxuan Chen, Jinyuan Chen, Heng Qu, Kun Shao, Jian Luan
Abstract:
Personal AI is moving beyond chat‑only interaction toward continuous services that span phones, cars, homes, wearables, cameras, and tools. In this setting, memory cannot remain a cache of prior conversations. It should serve as a continuity and governance substrate: preserving durable user state, grounding answers in multimodal and device evidence, supporting correction and forgetting, bounding policy evolution, and remaining deployable under latency, cost, privacy, and edge‑cloud constraints. This technical report presents Mi‑Memory, a lifecycle memory framework for Personal AI organized around four roles: Structure, Expansion, Evolution, and Deployment. A shared audit contract links these roles through four recurring artifact families: typed evidence payloads preserve source identity and provenance, diagnostic traces localize evidence loss across the serving pipeline, strategy artifacts make memory‑policy changes explicit, and gate/rollback records bound accepted evolution. MiMemory instantiates the roles through MemStack, MemSense/MemFuse, D^2ACCI/E^2MEND, and LiteMem. In controlled‑reference Structure evaluations, MemStack reaches 93.59%, 57.24%, and 87.47% on LoCoMo, PersonaMem‑V2, and LongMemEval, respectively; other tracks report module‑level, preliminary/internal, transfer‑feasibility, or design‑only evidence with explicit boundaries. MiMemory is a step toward auditable, evidence‑gated, and deployment‑aware memory systems for Personal AI. Project homepage: https://darwin‑agent.github.io/Mi‑Memory/ .
Authors:Haodi Fan, Zucong Lan
Abstract:
Agent Skills have become persistent behavioral artifacts across independent AI agent systems. They combine natural‑language task specifications with metadata and optional references, scripts, assets, hooks, package manifests, tests, and companion interfaces. Existing studies explain how Skills are specified, executed, maintained, and evolved, but lack an ontology that defines these artifacts as independent software objects. This paper introduces Skillware as the software abstraction that extends software engineering to persistent Behavioral Artifacts in agent systems. A Skill Artifact specifies reusable task behavior; a Skillware Unit manages that artifact as software through an independent identity and lifecycle. A compatible Agent Host activates the unit for runtime interpretation. Three necessary conditions operationalize category membership: behavioral primacy, independent software identity, and an Agent Host execution relationship. Lifecycle Continuity records whether the same unit identity persists through update, maintenance, rollback, and removal as a separate software‑grade property. Evidence combines the Agent Skills specification, a frozen corpus of 138,133 content‑deduplicated SKILL.md records associated with 20,556 repository identifiers, independent empirical studies, 15 category‑boundary cases, and 13 fixed‑revision engineering implementations. The evidence establishes a recurring artifact envelope, separable software identities, documented or reconstructed activation paths, and lifecycle engineering pressure. Skillware provides the software ontology and engineering lifecycle through which agent capabilities can become identifiable, composable, and maintainable software artifacts with an explicit basis for future evolution. Public design‑pattern and evidence materials are available at https://github.com/MetaInFLow/skillware‑patterns.
Authors:Gioele Nanni, Christopher Lee
Abstract:
A walking fly steers toward a goal direction, held as a bump of activity across the FC2 neurons of the fan‑shaped body. These neurons also inhibit one another over distance, more strongly the farther apart they are, a feedback proposed to keep the fly on a single goal. We asked, from the connectome, what circuit produces this inhibition, and whether it lets FC2 actively choose one goal among competitors (a winner‑take‑all) or simply keeps a goal set elsewhere as one clean bump. Tracing the wiring in a single FlyWire brain, we find the inhibition is almost entirely global: four FB5A cells inhibit every FC2 neuron roughly equally, with a smaller, distance‑dependent contribution from hDelta interneurons and a negligible direct component. A ring‑attractor winner‑take‑all (the kind the compass uses) requires local recurrent excitation that the FC2 wiring lacks, so this geometry cannot build one; and across a range of dynamical models, including a spiking network, no version of the circuit locks onto a winner at the connectome‑scaled reference coupling. FC2 therefore normalizes an externally set goal rather than selecting it, with FB5A likely acting as the global normalizer, much as the APL neuron does in the mushroom body. We are explicit about two open points: a different mechanism, mutual inhibition between two competing goals (which hDelta supplies), could in principle select at very strong coupling, and we bound rather than exclude it; and FB5A's inhibitory identity is a low‑confidence prediction of the connectome's transmitter classifier, not yet measured, and likely not GABAergic. We then ask where the goal is actually set: the connectome nominates an upstream hDelta network and rules out the leading proposed alternative, whose neurons supply under 0.2% of FC2's input. Finally, we propose a direct experiment, silencing FB5A while imaging FC2, that would test the account.
Authors:Sanket Sharma
Abstract:
We present NGPS (Next‑Generation Positioning System), a visual geo‑localization framework for high‑altitude UAVs that provides GPS‑free absolute positioning by matching down‑facing images to georeferenced satellite imagery with deep features. The system combines (1) adaptive confidence‑weighted UKF fusion, where NGPS covariance is modulated by RANSAC inlier ratio, reprojection error, and match confidence; (2) velocity‑predictive kernel extraction, using VIO velocity to predict the satellite search region; and (3) an asynchronous multi‑rate temporal priority queue that interleaves absolute position (1‑2 Hz), VIO (10‑20 Hz), and IMU (100‑200 Hz) updates in chronological order. Globally optimized poses from VINS pose‑graph optimization, anchored by NGPS corrections, further enable real‑time 2.5D georeferenced orthomosaic reconstruction. On five flight sequences (60‑150 m AGL), NGPS achieves 2.94 m position RMSE, with worst‑case ATE 6.04 m at 150 m AGL and 2 m/s, yielding a 3.5x improvement over standalone monocular VIO. The system runs in real time on an NVIDIA Jetson Orin NX. Part of the implementation is open‑sourced at https://github.com/snktshrma/ngps_flight.
Authors:Bo Liang, Chen Gong, Wei Gao, Chenren Xu
Abstract:
Millimeter‑wave (mmWave) radar enables privacy‑friendly human sensing, but its sparse point clouds are physical measurements of view‑dependent electromagnetic reflections and only indirectly characterize body articulation. Recovering a complete 3D pose from such partial, geometry‑dependent observations is therefore under‑constrained. Existing methods directly regress joint coordinates from paired radar‑pose data, relying on the same limited paired supervision to learn radar perception, human‑body structure, and their alignment. This coupling can encourage dataset‑specific shortcuts under ambiguous radar observations. We propose Wave2Body, a radar‑to‑body token translation framework that decouples these learning targets using a self‑supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them. Experiments on M4Human and mmBody show that Wave2Body achieves stronger cross‑domain generalization than previous methods while incurring much lower computational costs for training and inference. All the code and experiment results are publicly available at https://github.com/Galaxywalk/Wave2Body.
Authors:Haozhe Jia
Abstract:
Large language models leak parametric knowledge of realized outcomes into historical financial decision tasks. Existence is settled; what users lack is a cheap way to audit a given model for it. We present HindsightBench, a black‑box behavioral audit protocol that profiles parametric hindsight in any time‑indexed LLM decision task at probe‑level cost (no backtests, no logprobs, no corpus access). The protocol chains a four‑arm date‑manipulation matrix (revealed/date‑only/masked/transplanted), dual memory probes (date recovery; outcome recall), and six per‑model metrics ‑‑ trigger strength, transplant effect, post‑cutoff placebo, recoverability, behaviorally effective knowledge cutoff, and a recall‑accuracy dissociation coefficient ‑‑ with explicit gates where identifiability is data‑dependent. Applying it to 15 models from seven vendors on a 258‑node vintage‑correct macro panel yields three headline patterns: (i) the date‑trigger reflex tracks training generation, not scale ‑‑ absent across the 2024 open‑weight generation from 1B to 70B, present in every tested 2026‑generation model, and switching on within one vendor lineage (Qwen3 ‑> Qwen3.6) at fixed MoE architecture and 3B active parameters; (ii) effective cutoffs span 22 months across vendors and precede vendor‑reported dates by up to eight months, invalidating calendar‑window placebo designs; (iii) audit results are not invariant to serving ‑‑ BF16 serving of an FP8‑referenced model breaks the trigger estimate's stability while AWQ‑INT4 preserves it, and a provider‑locked reasoning regime makes one probe non‑convergent ‑‑ so the protocol ships with operational requirements (pin quantization and thinking regime; disclose parser and sampling policy). We release the panel, frozen preregistrations, per‑model audit rows with measured dollar costs, transcripts, and one‑command regeneration.
Authors:Bohan Su, Jiashuo Wang, Fangyi Liu, Mang Ye
Abstract:
Universal person re‑identification (ReID) aims to retrieve pedestrian identities across diverse real‑world scenarios, including severe occlusions, clothing changes, and cross‑modality shifts, within a unified model. However, existing 2D representations fundamentally struggle with spatial ambiguities due to a lack of depth and topological awareness, while naively introducing monocular 3D priors often causes severe negative transfer due to geometric estimation noise under extreme visual degradation. To safely harness the clothing‑invariant and canonical structural properties of 3D geometry, we propose UniGeo, a Universal Monocular 3D‑Enhanced ReID framework driven by a Consistency‑Aware Reliability Gate and Dual‑Stream Residual Fusion. Specifically, the processing of 3D information is strategically decoupled into geometric extraction and dynamic utilization. To provide pure structural compensation, we project monocular 3D parameters into kinematic joint representations, explicitly capturing instance‑level geometric topology to resolve appearance‑based ambiguities. To robustly incorporate these cues without perturbing the reliable 2D feature space, we isolate the 3D prior as a late‑stage structural residual; modulated by the consistency‑aware gate, this mechanism adaptively filters geometric noise and enables controlled fallback to the pure 2D baseline. Extensive experiments show that our method improves challenging, structure‑sensitive scenarios while preserving competitive performance on clean domains. Code is available at https://github.com/BohanSu/UniGeo.
Authors:Tianyue Jiang, Yanlin Wang, Xin He, Daya Guo, Jiachi Chen, Ming Wen, Ensheng Shi, Xilin Liu, Yuchi Ma, Guanbin Li
Abstract:
While Large Language Models have greatly advanced automated issue resolution, existing agent‑based methods exhibit a fundamental limitation in their insufficient exploration of repair strategies. This insufficiency manifests in two key aspects. First, the exploration of multiple potential edit locations is limited. Second, the exploration of repair attempts at each location is also insufficient. To address these challenges, we present PhoenixRepair, a multi‑agent framework that systematically explores multiple candidate edit locations and performs iterative reflection and refinement on patch generation, thereby expanding the search space of repair strategies. Our framework begins with multi‑location sampling, optionally augmented with graph‑based localization information for difficult tasks, followed by iterative reflection and refinement to generate better patches, culminating in final‑round generation guided by distilled insights from all historical attempts. Experiments on SWE‑bench‑Verified demonstrate that PhoenixRepair achieves the largest relative improvement of 7.8% over SWE‑agent under DeepSeek‑V3.1, and attains the highest resolved rate of 76.0% Pass@1 under MiniMax‑M2.5. Meanwhile, it achieves higher fault localization accuracy than existing approaches. Our code is available at https://github.com/DeepSoftwareAnalytics/PhoenixRepair.
Authors:Zhiqiang He, Zhi Liu
Abstract:
For decades, ABR has kept two kinds of intelligence apart. Neural policies learn rich behaviors yet forget them the moment the environment changes; rules never learn, and never forget. Every prior attempt to combine them has kept this separation, letting rules supervise, constrain, or override the network from outside. We dissolve the boundary itself. But no union can be trusted before it can be tested, and ABR has never known how to measure what its policies learn or forget. The field's yardstick is bandwidth statistics, and we show it misleads. Identical statistics can hide entirely different outcomes, while wildly different statistics can hide similar ones. We replace the yardstick before building the bridge, with Texture‑Aware Generalization Evaluation, a protocol that judges a policy by its whole training journey across traces whose temporal nature is laid bare. What truly breaks a policy is invisible. No statistic reveals it, no feature extracts it, yet rules walk through it untouched, for they reason from physics and owe the data nothing. So we build the bridge. Neuro‑Symbolic Manifold Alignment (NSMA) embeds rule decisions as anchors inside the latent space of the neural policy, so that it keeps learning where learning pays, and can no longer forget what rules have always known. Generalization cannot be argued, only survived. We raise NSMA on 3G traces alone and release it, without fine‑tuning, into eight unseen datasets spanning 4G, 5G, and WiFi, and onto a real‑world player. It outperforms every state‑of‑the‑art baseline. And when we open its latent space to ask why, probing and visualization return the same answer the design promised. https://tinyzqh.github.io/NSMA/
Authors:Koyar Afrasyab
Abstract:
Readiness stress‑testing of medical AI has focused on closed‑ended and multimodal benchmarks. We extend it to open‑ended clinical conversation under missing information, where safe behavior means recognizing absent information and qualifying, clarifying, or not over‑committing ‑ and where the evaluator becomes part of the measurement. We stress‑test four models ‑ three flagships (Claude Opus 4.8, GPT‑5.5, Grok 4.3) and one mid‑tier model (Gemini 3.5 Flash) ‑ by deleting the latter half of the final user turn in HealthBench conversations, grading responses with a four‑provider LLM‑judge panel and a blinded clinician‑anchored reference. Two evaluator‑facing results are robust. First, judge choice materially changes apparent safety: inter‑judge agreement is only moderate (Fleiss' kappa = 0.65), and after adjusting for each judge's general leniency (vote‑level logistic regression), a positive same‑provider association remains (exact permutation p = 0.04; GPT‑5.5 ~ +0.10 on the probability scale) ‑ large enough to change which model appears to over‑commit least once its own‑provider judge is excluded. Second, LLM judges are more permissive than clinicians on a blinded 50‑item subsample: all four are significantly more lenient than the stricter independent clinician (crediting appropriate uncertainty on 66‑84% of items vs 52%), and three of four than the author‑influenced consensus (Grok directional only; judge‑vs‑consensus kappa = 0.20‑0.43). On the author‑audited clinical‑underdetermined subset the permissiveness gap widened and the point‑estimate model ordering held. A closed‑ended MedQA anchor confirms accuracy is high and option‑order effects are within a +/‑5‑point equivalence region for three of four models, so the safety gap is about calibration, not knowledge. We release the harness, prompts, per‑item outputs, judge panel, perturbation audit, and human‑annotation protocol.
Authors:Binglu Wang, Sensen Niu, Ying Chen, Guangyu Guo
Abstract:
Gaze Object Prediction (GOP) aims to localize and recognize the objects humans attend to, a task crucial for understanding human‑centric interactions. However, existing methods are typically trained under a closed‑vocabulary paradigm with a fixed label space and evaluated on scene‑specific datasets, limiting their applicability to real‑world scenarios where gaze targets often follow a long‑tail distribution or belong to unseen categories. To address this gap, we introduce Diverse Scenes for Gaze object prediction (DiSG), a benchmark containing 86 in‑the‑wild categories that facilitates the evaluation of Open‑Vocabulary GOP (OVGOP). Building on DiSG, we propose a framework that leverages text‑driven object discovery to localize potential gaze candidates, with a gaze‑guided selection module to pinpoint the intended target from the candidate objects. Furthermore, to better capture semantic knowledge across diverse in‑the‑wild categories, we introduce Gradient‑Informed Selection Tuning (GIST) to selectively update parameters most relevant to a given class vocabulary. Extensive experiments demonstrate that our proposed model performs effectively in open‑vocabulary settings and also outperforms existing methods in the conventional closed‑vocabulary setting. The benchmark and code is available at https://github.com/sensniu/ovgop.
Authors:Ziming Wang, Yinghua Yao, Changwu Huang, Ke Tang, Xin Yao
Abstract:
Chain‑of‑thought (CoT) reasoning is widely used to improve both the performance and interpretability of large language models (LLMs), yet the generated reasoning may not faithfully support the final answer. We study this problem from a causal perspective, where a faithful CoT process should follow the chain Z\rightarrow X\rightarrow Y, with Z, X, and Y denoting the instruction, reasoning chain, and final answer, respectively. In this process, the instruction should affect the answer only through the reasoning chain. However, conventional autoregressive LLMs condition answer generation on both the instruction and the CoT, which still allows a direct instruction‑to‑answer shortcut. To address this issue, we propose CASE, a framework that combines training‑time causal alignment and inference‑time structural enforcement. During training, CASE builds counterfactual‑CoT, biased‑instruction, and empty‑instruction datasets, and applies selective‑loss fine‑tuning to strengthen CoT‑to‑answer dependence while suppressing instruction shortcuts. During inference, CASE masks direct attention from instruction tokens to answer tokens, preventing the model from bypassing the generated CoT. We provide an information‑theoretic analysis showing how these components promote faithful chains. Experiments on three models and four benchmarks show that CASE achieves a 37% average per‑setting relative improvement in overall CoT faithfulness over the strongest baselines, exhibits stronger cross‑dataset faithfulness transfer, and maintains competitive average accuracy. Code is available at https://github.com/oddwang/CASE.
Authors:Daisuke Kikuta
Abstract:
This paper proposes AI Tour Meeting, a group travel planning framework powered by multiple Large Language Model (LLM)‑based agents. The agents are instantiated with distinct personas and collaboratively seek an itinerary that satisfies their constraints and preferences through natural language discussion. The framework enables easy and flexible orchestration of such discussions by providing interfaces for configuring agent personas, discussion workflows, monitoring, and LLM deployment. Its primary use case is a simulation tool for analyzing the behavior of multiple LLM agents during tour planning discussions. This paper demonstrates the utility of the framework by presenting system validation and several analytical results obtained by the framework.
Authors:Jiayu Ding, Meilu Song, Xiaoyi Zhang, Hongbo Jin, Yichen Jin, Xiangtian Si
Abstract:
Recent advancements in 3D Gaussian Splatting (3DGS) have enabled language‑guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single‑target queries. To reflect the ambiguity of real‑world instructions, we introduce the Generalized Referring 3D Gaussian Splatting Segmentation (GR3DGS) task, which requires dynamically segmenting an arbitrary number of targets (0, 1, or N). To facilitate comprehensive evaluation of this new task, we construct two new benchmarks: GR‑LERF and GR‑ScanNet. Crucially, existing R3DGS paradigms exhibit fundamental technical bottlenecks that severely limit their performance on the GR3DGS task: they lack intrinsic 3D point‑level understanding by operating merely on 2D rendered pixels, and they incur prohibitive computational overhead by requiring per‑scene optimization to embed heavy semantic features. To dismantle these bottlenecks, we propose ZeroSplat, a novel training‑free and zero‑feature framework. ZeroSplat lifts 2D Vision‑Language Model (VLM) priors into 3D space through robust multi‑view geometric constraints. This strategy enables intrinsic point‑level understanding without incurring any additional feature storage. Extensive experiments demonstrate that ZeroSplat significantly outperforms state‑of‑the‑art methods across generalized and single‑target scenarios while maintaining exceptional efficiency. Project Page: https://inkmind‑ai.github.io/ZeroSplat
Authors:Xin Ming, Yuxuan Han, Junhai Yong, Feng Xu
Abstract:
Reconstructing high‑fidelity facial geometry with an assigned topology is essential for digital avatar creation and animation, yet existing automated methods often trade off geometric fidelity and in‑the‑wild generalization. We present UVFaceFusion, a feed‑forward framework for multi‑view, fixed‑topology face reconstruction from daily images. Our key idea is to replace heuristic topological optimization with learnable neural fusion in a canonical UV space. Given multi‑view images, we first obtain dense point maps and facial UV correspondences of each view using VGGT and Pixel3DMM, respectively. Then, the view‑specific point maps are lifted into the canonical UV domain and fused with a novel mask‑aware neural fusion network. The network predicts a complete UV‑space point map, from which a fixed‑topology mesh is directly sampled. Although trained only on Ava‑256, UVFaceFusion generalizes well to multiple public benchmarks and in‑the‑wild captures, benefiting from its canonical UV‑space geometry‑to‑geometry fusion that reduces dependence on dataset‑specific appearance and capture conditions. Experiments on various benchmarks show that UVFaceFusion achieves state‑of‑the‑art reconstruction accuracy while reconstructing a mesh from 16 input views in less than 3 seconds on a single RTX 4090. Code is available at https://github.com/grignarder/UVFaceFusion.
Authors:Jie Luo, Qi Jin, Xinming Zhang
Abstract:
Multi‑modal Sequential Recommendation (SR) incorporates rich side information (e.g., textual and visual features) to enhance dynamic user preference modeling. However, existing frameworks inevitably suffer from a Dual‑Noise Dilemma: (1) Feature‑level redundancy stemming from the semantic gap between generic pre‑trained representations and fine‑grained recommendation intent; and (2) Sequence‑level stochasticity induced by spurious interactions such as accidental clicks. To break this bottleneck, we propose DDMSR, a novel Dual‑level Denoising Multi‑modal Sequential Recommendation framework that systematically purifies signals from both feature‑topological and sequence‑frequency perspectives. Specifically, we first design a graph‑based feature denoising module that leverages Laplacian smoothing on item semantic graphs as a structural low‑pass filter, effectively suppressing high‑frequency semantic noise while preserving salient features. For sequence purification, we introduce a frequency‑domain sequence denoising module, utilizing the Fast Fourier Transform and a learnable frequency filter to adaptively modulate the interaction spectrum and attenuate anomalous signals. Furthermore, a multi‑modal contrastive alignment objective is incorporated to bridge the heterogeneity gap and enforce cross‑modal semantic consistency. Extensive experiments on four public benchmark datasets demonstrate that DDMSR consistently outperforms state‑of‑the‑art baselines, providing a highly robust and efficient solution for multi‑modal sequential recommendation. The source code is available at: ~\hrefhttps://github.com/jluo00/DDMSR\textcolorbluehttps://github.com/jluo00/DDMSR.
Authors:Jinying Xiao, Bin Ji, Shasha Li, Xiaodong Liu, Ma Jun, Jiacheng Jie, Chao Wang, Nyima Tashi, Jie Yu
Abstract:
As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution. Existing retrievers often treat skill descriptions as ordinary documents, overlooking their highly regular structure: shared descriptive patterns recur across many skills while providing little evidence for distinguishing the required capability. We show that this shared descriptive background systematically contributes to dense relevance scores, induces a pronounced energy gap between queries and skill documents, and obscures task‑relevant signals. Based on this observation, we propose SkillSight, a training‑free retrieval framework that calibrates shared background in both semantic and lexical spaces. Semantic Background Calibration estimates a background subspace from generic tokens identified by IDF, reducing similarity induced by shared descriptive patterns, while Lexical Evidence Calibration downweights shared background tokens to recover discriminative token‑level evidence. Experiments on SRA‑Bench and SkillBench‑Supp demonstrate consistent improvements across retrieval metrics, with SkillSight improving Recall@10 by up to 20.21 percentage points over the original dense retriever. In end‑to‑end evaluation, SkillSight achieves the best overall performance across three agent models and outperforms LLM Selection by up to 4.97 percentage points. It is also up to 1,248 times faster than the Dense + Reranker baseline. These results identify shared descriptive background as a key source of bias in skill retrieval and demonstrate that explicitly calibrating it enables accurate and efficient skill selection without additional training. Our code is available at https://github.com/xiaojinying/SkillSight.
Authors:Yun Xiao, Zhihong Hong, Jiandong Jin, Chenglong Li, Jin Tang, Amir Hussain
Abstract:
Unmanned Aerial Vehicle (UAV) object tracking has emerged as a popular research field with broad practical applications. Modern UAVs are increasingly equipped with both visible light and thermal infrared sensors. However, due to constraints in communication bandwidth, computational resources and power consumption, current systems often activate one modality and switch between modalities to maintain robust tracking in complex scenarios. Such modality switch inevitably leads to significant appearance change and sudden spatial shift, posing great challenges for existing tracking algorithms. To handle this problem, we propose a novel State‑Aware Representation Learning Approach called SARLA, which perceives the inconsistent modality states of current frame with template and last frame in the target representations to adapt to the sudden changes in both appearance and position, for robust cross‑modal object tracking. In particular, we propose the Modality State Aware Representation Module (MSARM) and Spatial State Aware Representation Module (SSARM). MSARM guides the model to learn appearance correlation, bridging the modality gap, while SSARM models cross‑frame spatial correlation to mitigate sudden spatial shift impacts. In addition, we design a spatial shift prediction loss to further handle the effects of spatial variation caused by modality switch. To promote the development of this research field, we establish a large‑scale video benchmark called CM‑UOT, which consists of 1079 cross‑modal sequences with an average video length greater than 621 frames and encompasses over 671K frames in total. Extensive experiments on CM‑UOT dataset demonstrate the superior performance of the proposed SARLA against 20 excellent tracking methods. The source code, datasets, and evaluation protocols associated with this work are publicly available at: https://github.com/hongsmile365/sarla‑.
Authors:Zihan Zhang, Yu Bao, Xiao Ding, Tianyi Jiang, Kai Xiong
Abstract:
Translating brain signals into text could restore communication for people with severe paralysis, yet practically usable systems to date rely on invasive electrocorticography (ECoG). Electroencephalography (EEG) offers a non‑invasive alternative, and EEG‑to‑text (EEG2Text) has been widely explored. Interestingly, however, EEG2Text models generally rely on teacher‑forcing evaluation; without it, they fail to generate meaningful decoding. This reliance prevents EEG2Text from being applied in real‑world, non‑academic settings. This has fueled numerous debates about whether EEG2Text is a meaningful direction, by extension, and whether EEG truly contains decodable linguistic information. Here, using a neuropsychology‑informed paradigm, we find that existing EEG2Text benchmarks have neglected EEG instability, a flaw that has confounded inference and sparked debate. Our experiments furnish key evidence for the feasibility of teacher‑forcing‑free EEG2Text decoding. Accordingly, we assemble the Corpus OF Eeg‑To‑Text (COFETT) using a 128‑channel high‑density EEG cap, providing a benchmark dedicated to evaluating EEG2Text models. In comparisons with multiple existing benchmarks, COFETT achieves SOTA ability to distinguish among model performances and enables robust, teacher‑forcing‑free evaluation, thereby opening a path toward practical EEG2Text applications. COFETT is open sourced in https://github.com/baoyudu/COFETT.
Authors:Robert James Brock, Sebastian Maximilian Krupa, Jason Kahei Tam
Abstract:
The FathomNetCLEF 2026 competition combines underwater object detection and fine‑grained marine species classification under a positive‑unlabeled evaluation setting. The provided training labels are sparse, while the hidden test set is out‑of‑distribution relative to the training imagery, creating both annotation incompleteness and source‑shift challenges. We describe DS@GT ARC's multi‑stage system developed for this setting while keeping model training restricted to the data provided by the competition. The final private‑leaderboard model uses a frozen Megalodon YOLOv8x detector as a class‑agnostic proposal generator, combines global and tiled inference with tile‑edge filtering, classifies expanded proposal crops with a LoRA‑finetuned DINOv3 ViT‑H classifier, and ranks predictions using weighted geometric fusion of detector and classifier confidence. This system placed 12th out of 102 teams. A closely related variant added a locally trained TTN‑inspired validity head as a light reranking signal, improving public‑leaderboard and proxy‑evaluation performance but slightly reducing private‑leaderboard performance. Across experiments, the strongest lesson was that train‑derived validation and detector‑only metrics were not reliable enough for model selection. Instead, we used proxy datasets only for validation and comparison, and combined those signals with leaderboard feedback and targeted ablations. These experiments showed that reserving proposal recall, avoiding over‑aggressive filtering, and improving downstream ranking were more effective than fine‑tuning the detector or directly training on noisy pseudo‑labels. Code: https://github.com/dsgt‑arc/fathomnetclef‑2026.
Authors:Hao Tang, Songyun Xie, Xinzhou Xie, Can Liao, Xin Zhang, Bohan Li, Zhongyu Tian, Dalu Zheng
Abstract:
Most existing EEG‑based emotion recognition studies formulate affective decoding as static category prediction, although emotions elicited by continuous stimulation evolve over time, accumulate, reach peak intensity, and then recover. This motivates EEG‑based dynamic affective trajectory prediction, which estimates continuous affective intensity curves from sequential EEG observations. Existing temporal regression models can capture coarse intensity trends but often fail to preserve peak‑centered structure, leading to inaccurate peak timing and terminal‑peak bias, where the predicted maximum is shifted toward the end of a trial. To address this issue, we propose PeakFlow, a peak‑guided coarse‑to‑refined framework for EEG‑based dynamic affective trajectory prediction. PeakFlow first learns a coarse affective flow through EEG temporal tokenization and masked temporal modeling, then applies a lightweight residual refiner for peak‑guided bounded calibration. The refiner uses trajectory‑aware cues and a peak‑centered objective combining global trajectory consistency, peak‑zone emphasis, peak‑probability localization, terminal suppression, and residual regularization. This design preserves the global affective trend while correcting peak misalignment, peak‑value deviation, and false‑terminal predictions. Leave‑one‑subject‑out experiments on SEED‑VII show that PeakFlow improves both global trajectory fitting and peak‑centered temporal reliability over strong dynamic modeling baselines. Auxiliary evaluation on FIRMED further suggests its potential for sparse peak‑centered ordinal intensity analysis. These results highlight the importance of peak‑aware modeling for temporally faithful EEG‑based dynamic emotion prediction. Code is available at https://github.com/jukebox333/PeakFlow.
Authors:Abdul Basit Tonmoy, Kazi Fardinul Hoque, Md. Shahrier Islam Arham, Arman Luthra
Abstract:
A single embedding space that covers text, images, video, and audio lets one index serve every query a user can pose. Embedding models built on vision‑language backbones now lead text/image/video retrieval benchmarks but lack audio entirely, while audio‑text retrieval is led by specialist systems that serve no other modality. We present the Fusion Embedding family, which adds audio to a frozen vision‑language embedding base whose parameters are never updated: generation 1 (fusion‑embedding‑1) trains only a 16.4M‑parameter connector between a frozen audio tower and the frozen base, and generation 2 (fusion‑embedding‑2) adds modality‑gated deep adapters (44.2M parameters) whose branch never executes on text, image, or video inputs: their outputs are bit‑for‑bit those of the released base, verified after every training run. Because the base already binds text, images, and video, aligning audio to text alone makes audio‑image retrieval emerge, with zero paired audio‑visual training data. Alongside the recipe we map its design space with controlled negative results (rewriting training captions with an LLM, substituting a leaderboard‑stronger audio tower, and widening the connector each reduce retrieval) and with training‑protocol findings that we expect to transfer to any frozen decoder‑LM embedding backbone. Both generations train in hours on a single GPU. Weights, code, and the evaluation harness are openly released.
Authors:Yuxiang Ji
Abstract:
Mined code corpora are abundant but uncontrolled: a snippet's semantics, surface "messiness," and difficulty are whatever the wild contained; there is no known‑optimal reference to grade against; and any public sample may already sit in a model's training set. We present Spaghetti Architect, a tool that mints code datasets with the control such corpora lack. An anti‑optimization transpiler maps a clean, language‑agnostic JSON intermediate representation to deliberately redundant, fully‑flattened programs in five languages (Python, JavaScript, Go, Java, C++); every program is compiled, run, and checked against a reference oracle, so each instance is correct by construction. The clean IR is a known‑optimal reference, messiness is dialed by strictly‑nested anti‑pattern profiles, each instance is labelled along two orthogonal difficulty axes, intrinsic (problem size) and incidental (presentation at fixed semantics), and contamination is resisted by minting fresh variants from a private held‑out seed. We give construct‑validity evidence that the quality order moves established complexity and readability metrics, and report baselines on a four‑model open ladder: exact match rises with scale, and the intrinsic knob collapses arithmetic‑aggregation accuracy of even the strongest model to zero. Further, development‑set scores equal freshly re‑minted held‑out counterparts within |Δ|\le 0.012 (comprehension) and \le 0.011 (refactoring); on identical programs, refactoring equivalence (0.73 \rightarrow 0.99) is scale‑invariant while output prediction collapses; and ablating the generator's self‑annotations shows they inflate the weakest model an order of magnitude more than the strongest (‑0.173 vs ‑0.017): the annotated ladder resolves one of three adjacent pairs where the unannotated resolves all three. Open source (MIT), dependency‑free, archived under a persistent DOI.
Authors:Tomohiro Kikuchi, Kohei Yamamoto, Yukihiro Nomura, Yosuke Yamagishi, Takeharu Yoshikawa, Toshiaki Akashi, Jun Kamohara, Hiroyuki Fujii, Harushi Mori
Abstract:
Purpose: To develop and validate a deep learning ensemble for estimating adult sex, age, height, and weight from coronal digitally reconstructed radiographs (DRRs) generated from diagnostic CT. Materials and Methods: This retrospective study included 128,621 CT examinations from 80,004 adults at nine institutions in Japan. Three multitask models‑ConvNeXt‑Base, ViT‑Base/16, and MaxViT‑Base‑were fine‑tuned using coronal DRRs and combined by weighted averaging. Data were split by institution into training (114,147 examinations; seven institutions), tuning (4,305; one institution), and test (10,169; one institution) sets; generalizability was assessed on two non‑Japanese datasets. Accuracy and mean absolute error (MAE) were used to evaluate sex classification and age, height, and weight regression, respectively. Body surface area (BSA)‑corrected heart and liver volume trends were compared using true versus estimated height and weight. Results: In the test set (median age, 69.9 years; 4,899 of 10,169 [48.2%] male), overall sex‑classification accuracy was 0.997 (95% CI, 0.996‑0.998), and MAEs were 3.57 years (3.51‑3.63), 2.59 cm (2.54‑2.64), and 3.40 kg (3.34‑3.47) for age, height, and weight, respectively. In examinations covering the chest through pelvis, accuracy was 1.000, and MAEs were 3.15 years, 2.28 cm, and 3.18 kg, respectively. BSA calculated from estimated values reproduced age‑related heart and liver volume trends obtained using true values. On non‑Japanese datasets, height error increased but was reduced by continued fine‑tuning. Conclusion: The ensemble estimated adult sex, age, height, and weight from CT‑derived DRRs, with generally lower errors in examinations with broader anatomical coverage.
Authors:Y Huynh, Duc Thanh Nguyen, Mohamed Abdelrazek
Abstract:
The relationship between object perception and reconstruction is well established in human vision, yet remains underexplored in computer vision. In this paper, we demonstrate that learnt object perception can significantly enhance 3D reconstruction. Focusing on the challenging task of single‑view 3D object reconstruction, we propose a method that leverages perceptual signals extracted from pretrained perception models capturing semantic and geometric information to drive the reconstruction of an object from its single image. Our approach is model‑agnostic and can be integrated into various reconstruction methods in a plug‑and‑play manner. Experiments with two state‑of‑the‑art single‑view 3D reconstruction pipelines in a benchmark dataset show consistent and substantial improvements achieved by our method, validating the effectiveness of incorporating perception into generation. We provide in‑depth analysis of various aspects of our method and its application. Our project page is at https://ynhuhuynh.github.io/perception‑3d/.
Authors:Yongsen Zheng, Ruilin Xu, Guohua Wang, Liang Lin, Kwok-Yan Lam
Abstract:
The Matthew effect is a big challenge in Recommender Systems (RSs), where popular items tend to receive increasing attention, while less popular ones are often overlooked, perpetuating existing disparities. Although many existing methods attempt to mitigate Matthew effect in the static or quasi‑static recommendation scenarios, such issue will be more pronounced as users engage with the system over time. To this end, we propose a novel framework, Multi‑Hypergraph Boosted Multi‑Interest Self‑Supervised Learning for Conversational Recommendation (HiCore), aiming to address Matthew effect in the Conversational Recommender System (CRS) involving the dynamic user‑system feedback loop. It devotes to learn multi‑level user interests by building a set of hypergraphs (i.e., item‑, entity‑, word‑oriented multiple‑channel hypergraphs) to alleviate the Matthew effec. Extensive experiments on four CRS‑based datasets showcase that HiCore attains a new state‑of‑the‑art performance, underscoring its superiority in mitigating the Matthew effect effectively. Our code is available at https://github.com/zysensmile/HiCore.
Authors:Sam O'Nuallain, Nithya Rajkumar, Ramya Narayanasamy, Hanna Jiang, Shreyas Chaudhari, Andrew Drozdov
Abstract:
We present AutoIndex, a framework for learning representation programs: executable transformations that map raw documents into the representations exposed to a retrieval system. Rather than tuning retrievers, rerankers, or a small set of preprocessing hyperparameters, AutoIndex searches over programs that slice, enrich, normalize, reweight, or reorganize documents before indexing. At each iteration, AutoIndex performs validation‑guided program search, in which agents diagnose failures of the current program and synthesize candidate updates, retaining only updates that improve retrieval quality under the resulting index. We evaluate AutoIndex on CRUMB, a benchmark of heterogeneous retrieval tasks, with BM25 held fixed across all experiments. The learned programs improve recall over a static full‑document BM25 baseline on all 8 tasks, with average gains of +8.4% in Recall@100 and +8.3% in nDCG@10, and largest gains of +30.5% in Recall@100 and +43.6% in nDCG@10. These results suggest that document representation should not be treated as a fixed preprocessing choice made before retrieval begins, but as an explicit optimization target. Code to reproduce our results is available at https://github.com/auto‑index/autoindex.
Authors:Abhidip Bhattacharyya, Shira Wein
Abstract:
Discourse relations provide document structure, critical to language understanding and enabling language model performance and ethicality. In this work, we investigate how instruction‑tuned Transformer models (LLaMA and Mistral) encode discourse relations in English, with a particular focus on the contrasting relations of causation and antithesis. Framing the task as a next‑token prediction task and applying a suite of interpretability techniques to test model internals, our findings show that certain early layers make predictive decisions at mid‑sequence tokens, while some mid‑level layers finalize their decisions closer to the last token. Most of the remaining layers primarily propagate earlier decisions rather than actively influencing them. Additionally, we observe that some layers exhibit a preference for one answer over alternatives, suggesting asymmetric representation of discourse‑based reasoning.\footnoteOur code is available at https://github.com/abhidipbhattacharyya/causation_vs_antithesis
Authors:Yen-Chi Cheng, Chen Gao, Chuhan Chen, Tuotuo Li, Rajvi Shah, Ayush Saraf, Changil Kim, Liangyan Gui, Alexander Schwing, Johannes Kopf, Hung-Yu Tseng
Abstract:
Novel view rendering of large and complex reconstructed scenes is becoming increasingly photorealistic. However, most reconstructions remain static and lack the ambient motion that makes environments immersive. We present AniGS, a method for scene‑level animation of 3D Gaussian Splatting (3DGS) reconstructions that adds subtle, distributed dynamics, e.g., vegetation motion, while preserving rigid structures. Unlike existing 3D animation techniques which are limited to object‑centric subjects or small regions, AniGS is designed for large, cluttered, navigable scenes. AniGS represents the scene with a canonical 3DGS and models motion using a time‑conditioned deformation field. To animate the entire scene, we leverage a pretrained video diffusion model and introduce an iterative dataset‑‑model update strategy that progressively expands viewpoint coverage and repeatedly updates camera‑fixed training videos using a render‑and‑refine scheme. To prevent artifacts from unintended motion in static areas, we further introduce a composed video‑to‑video refinement scheme that restricts motion to desired regions. Experiments on five real‑world, large‑scale outdoor scenes demonstrate that AniGS produces natural ambient dynamics and high‑quality novel view videos, enabling more immersive viewing experiences of reconstructed environments.
Authors:Abir Harrasse, Michael Lan, Hunar Batra, Fateme Hashemi Chaleshtori, Chaithanya Bandi
Abstract:
Reasoning‑specialized language models show large performance gains over base models, yet the internal changes responsible for improved multi‑step reasoning remain poorly understood. It is unclear whether reasoning fine‑tuning improves local token‑level competence or globally reorganizes how models structure inference over time. We address this question by modeling Chain‑of‑Thought reasoning as a switching dynamical system (SDS), in which internal representations evolve under discrete latent policy states. Our framework combines time‑aware contrastive representation learning with discrete regime discovery to recover latent policies from activation trajectories. Across four benchmarks and model scales from 1.5B to 32B parameters, reasoning‑fine‑tuned models exhibit richer latent‑policy organization than their base counterparts, characterized by more differentiated transition structure and model‑dependent changes in state utilization, persistence, and mixing. The recovered regimes exhibit functional specialization aligned with distinct reasoning stages, and extensive controls confirm that their structure is not explained by correctness, representation learning, or modeling priors, but depends on the coherent temporal organization of reasoning trajectories. Causal interventions further show that the regimes are functionally meaningful: state‑swap ablations reduce one‑step predictive fit, while transplanting reasoning dynamics into base models improves performance on challenging reasoning problems. Finally, SDS‑guided pruning of failure‑prone reasoning prefixes outperforms self‑consistency in 11 of 12 model‑dataset settings, with gains of up to 12.5 percentage points. Together, our results suggest that reasoning fine‑tuning globally reorganizes latent dynamics, offering a new lens for mechanistic analysis and process‑level control of reasoning models.
Authors:Jiabing Yang, Yixiang Chen, Yuan Xu, Qisen Ma, Tao Yu, Peiyan Li, Yingda Li, Yan Huang, Liang Wang
Abstract:
Preference over model‑generated emotion descriptions is emerging as a standard evaluation metric for multimodal emotion understanding, exemplified by the MER2026 MER‑Prefer track on EmoPrefer. Such benchmarks assume that predicting the preferred description requires grounded cross‑modal understanding of the video. We conduct a systematic shortcut audit of EmoPrefer using content‑blind probes. A simple logistic regression using only description length and generator identity, without processing the text, video, or audio, performs comparably to LoRA‑finetuned 7B text and audio‑visual judges (65.8 versus 66.8 WAF on EmoPrefer‑V2). Generator identity is recoverable from description text with 99.5 percent accuracy, every candidate pair contrasts two distinct generators, and the human preference labels agree with a fold‑exclusive per‑generator win‑rate prior on 66 percent of the evaluated pairs. When the human label conflicts with this prior, trained judges still follow the style prior on 63 to 80 percent of the pairs. On a length‑matched subset that neutralizes verbosity bias, the tested media configurations yield no statistically significant improvement, while an ODIN‑inspired diagnostic that decouples the style shortcut leaves its content head near chance. These results do not imply that human preferences are inherently stylistic or that the descriptions contain no emotional information. Instead, they show that the current scores can be reached without verifying either description against the video. We recommend source‑balanced pairing, strict length control, counter‑stereotypical sliced reporting, and multi‑annotator consensus for future cross‑generator evaluations. Code is available at https://github.com/jiabingyang01/EmoPrefer‑Audit.
Authors:Tapan Parikh
Abstract:
When a language model must choose one answer from a large space of equally valid options, a format clause ‑‑ "Reply with JSON only" ‑‑ changes which answer it chooses. We re‑run the One‑Word Census (arXiv:2607.12796): 31 wide‑answer‑space category prompts asked of 44 models, now with the reply requested in JSON ‑‑ no schema enforcement, no constrained decoding, only the request. Convergence deepens sharply: on the unconstrained "Pick a word" prompt the modal answer rises from 41% to 64% of the pool and distinct answers fall from 52 to 36; mean answer‑choice surprisal drops from 1.80 to 1.58 bits. The tax is progressive: six of 44 models move individually (BH‑FDR q=.10), all toward the mode, led by the most distinctive models, while the conformist floor is immobile. It is a sharpener, not a re‑indexer ‑‑ the plain‑chat modal answer survives in 28 of 31 categories. Defaults are register‑indexed: a within‑run re‑sample (n=20) finds JSON shifts 53% of a model's stable chat defaults, mostly back to the crowd, and installs defaults absent from chat (Claude Fable 5 answers "cerulean" for colour 0% of the time in chat, 100% in JSON). Full‑battery controls reveal a register gradient: compression is significant and specific to the answer‑delivery formats models are trained to speak (JSON ‑0.22 bits, p=.0002; XML ‑0.19, p=.002), absent for YAML and CSV, and reversed for an arbitrary bracket wrapper (+0.13, p=.009) ‑‑ weighing the mechanism toward tool‑use post‑training. Enforcing the schema at the decoder (response_format) compresses no further than the request (‑0.03 bits): the collapse lives in the model's response to the register, not the decoder. Structured output is how software consumes language models, and that surface is served by a measurably more homogeneous model than the chat surface on which models are evaluated, compared, and chosen.
Authors:Kaiyuan Tang, Chaoli Wang
Abstract:
Recent advances in differentiable Gaussian splatting have highlighted the potential of primitive‑based approaches as alternative scene representations for interactive, high‑quality, volume visualization (VolVis) of large datasets. However, the explicit nature of current primitive‑based methods, combined with isolated optimization for each VolVis scene, results in redundant, non‑compact representations. We present ECoNGS, an efficient compressive neural Gaussian splatting framework for VolVis scene representation. ECoNGS employs lightweight neural networks to dynamically predict implicit, editable Gaussian splats from explicit anchor points, effectively combining model compactness and parameter efficiency of implicit representations with high‑performance rendering of explicit primitives. We explore a joint learning strategy that clusters geometrically similar scenes and shares parameters across them, significantly reducing overall training time and model size while maintaining reconstruction fidelity. To achieve a more compact scene representation, we further compress the explicit anchor attributes using a neural entropy model that estimates their probability distributions, enabling compact storage via entropy coding. We systematically investigate Gaussian initialization strategies and propose a simple yet effective scheme tailored for VolVis scenes, improving reconstruction accuracy and accelerating convergence. We evaluate ECoNGS qualitatively and quantitatively across various univariate and multivariate VolVis scenes, highlighting its superior performance over prior methods in training time, reconstruction quality, and model size. In particular, compared with the prior method iVR‑GS, ECoNGS improves reconstruction quality by up to 2.2 dB in PSNR while reducing the model size by up to 6.1x and the training time by up to 5.9x. The code is available at https://github.com/TouKaienn/ECoNGS.
Authors:Sarunyu Thongjarast
Abstract:
The random cut is one of the most fundamental shuffles in card‑based cryptography: it rotates a sequence of face‑down cards by a secret amount. Under this shuffle, two sequences of cards are indistinguishable if and only if they are cyclic shifts of each other. This motivates the question of whether, given two sequences of cards, inserting cards at matching positions can make them indistinguishable. A previous study shows that such an insertion is always possible when any cards may be inserted, as long as the two words are permutations of each other. This paper considers a stronger restriction: if the cards are binary, carrying only 0 or 1, can we insert only 0s to make the sequences indistinguishable? We call two words 0‑cyclically equalizable if one can insert 0s into both sequences at matching positions so that the resulting words are cyclic shifts of each other. Our main result is that two binary words of equal length are 0‑cyclically equalizable if and only if they have equal Hamming weight, that is, the same number of 1‑bits. Since equal Hamming weight is clearly necessary, the content of the paper is to show that it is also sufficient. Our proof is constructive: we encode a pair of binary words as a single word over the four‑letter alphabet A, B, X, O, reduce equalizability to a simpler condition in this encoding, and build the required insertion explicitly.
Authors:Patrik Reizinger, Wieland Brendel
Abstract:
Large language models (LLMs) now routinely draft literature reviews and assist with academic writing, which means a higher risk of fabricated references: GPTZero found 53 papers with hallucinated citations among NeurIPS 2025's accepted set. Rule‑ and LLM‑based verifiers are emerging, but no shared benchmark compares them and gives detailed failure diagnostics. We close that gap with HALLMARK (Hallucination benchmark): 2,526 BibTeX entries spanning 14 hallucination types, three difficulty tiers, six diagnostic sub‑tests per entry, and a contamination‑resistant held‑out split. On it we evaluate a DOI‑lookup baseline, frontier LLMs zero‑shot, tool‑augmented agents, and our own rule‑based, co‑designed verifier bibtex‑updater. Across the benchmark one result is consistent: the false‑positive rate, not recall, decides whether a verifier is deployable. HALLMARK makes it concrete through three failure modes: agentic lookups buy recall but inflate false positives; at a venue‑realistic base rate, the order‑of‑magnitude spread in false‑positive rates (FPRs) ‑‑ not recall ‑‑ governs whether a verifier's flags are mostly true catches or mostly noise; and most LLMs over‑flag papers published past their training cutoff, where only the two latest‑cutoff models hold their false‑positive rate near in‑distribution levels (a signal we report as descriptive, since it is confounded with possible recall of those entries). Thus FPR is the deployment bottleneck, but an undetected fabrication remains the costlier error for the scientific record.
Authors:Zhen Yu, Yachao Yuan, Zixiang Peng, Muting Li, Thar Baker
Abstract:
In traffic accident risk prediction, most studies overlook the extra noise that could be incorporated when fusing temporal features into spatial features, and some models struggle to capture global correlations among spatial regions. To address these challenges, we propose a novel traffic accident risk prediction framework named MambaLSTM. First, we develop a squeeze‑and‑excitation temporal feature fusion module to integrate temporal information without compromising spatio‑temporal integrity. Second, we introduce a new patch embedding module for effectively capturing semantic relationships among spatially adjacent regions. Additionally, we introduce a Mamba block based on state‑space models to model global spatial semantics in urban regions. Finally, we propose a MambaLSTM unit to efficiently capture long‑ and short‑term temporal dependencies for identifying dynamic risk patterns. Extensive experiments on real‑world datasets demonstrate the proposed model's superiority over state‑of‑the‑art methods. The code is released at https://github.com/Zhenzovo/MambaLSTM.
Authors:Yiheng Liu, Chuhang Zheng, Peiliang Gong, Jingtao Liu, Daoqiang Zhang, Qi Zhu
Abstract:
EEG‑based visual decoding provides a non‑invasive pathway for interpreting visual semantics. However, existing methods often overlook the perceptual asymmetry between foreground and background in complex scenes, leading to background interference and semantic misalignment. EEG signals also exhibit rapid temporal dynamics and nonstationary spatial patterns, making it difficult to capture the time‑varying brain connectivity associated with focal visual attention. To address these limitations, we propose FSDBN, a unified framework for robust EEG‑visual decoding. FSDBN introduces Semantic‑Consistent Saliency Alignment to separate semantically relevant foreground regions from background noise under joint saliency and semantic constraints. It further employs Semantic‑Prior Dynamic Gating Foreground Fusion to adaptively regulate the contributions of foreground and background features. In parallel, EEG signals are modeled as adaptive spatiotemporal brain networks whose functional connectivity dynamically reorganizes to capture neural responses to salient foregrounds. Experiments on zero‑shot brain‑to‑image retrieval demonstrate that FSDBN achieves 69.0 percent top‑1 accuracy and 92.2 percent top‑5 accuracy, outperforming previous state‑of‑the‑art methods. Code is available at https://github.com/LiuYiheng1/FSDBN‑EEG.
Authors:Rahul Suresh Babu, Shashank Indukuri
Abstract:
Tool‑augmented language‑model agents execute multi‑step workflows over external systems, resolving an entity once and then acting on it across subsequent steps. Prior work shows that in single‑step actions, agents select the correct tool but bind it to the wrong entity 24‑26% of the time. We study what happens to entity bindings over time: do they stay correct, silently drift to a different entity, or, if wrong from the start, propagate and compound? We formalize binding drift (correct at step 1, wrong later) as distinct from error propagation (wrong at step 1, carried forward), and score them on disjoint workflow sets so the two cannot be conflated. In a controlled multi‑step testbed (200 workflows, 580 entity‑binding‑scored steps, four enterprise domains, eight model backends spanning small to frontier), we find: (1) under controlled error injection, an entity lock (the intuitive "persist the first binding" fix) amplifies wrong actions from 907 to 2,746 (3.0x; bootstrap 95% CI [2.8, 3.3]), because it faithfully carries the seeded wrong entity into every later step; (2) the amplification reaches 8.5x on the most affected model (Claude Opus 4.5); (3) a practical LLM‑based re‑verifier (a single cheap second model call re‑reading the original instruction) reduces wrong actions by 79% (0.21x; CI [0.18, 0.25]), closing the gap to within 1 percentage point of an oracle upper‑bound (0.20x); and (4) in the natural (non‑injected) setting, baseline agents drift on 18% of eligible workflows, with the per‑step error rate rising across steps. Persistence and re‑verification are not interchangeable: a defense that eliminates drift can worsen propagation, and a practical re‑verifier nearly matches oracle recovery.
Authors:Hoang-Thang Ta
Abstract:
In recent years, Kolmogorov‑Arnold Networks (KANs) have attracted increasing attention due to their effectiveness in machine learning and scientific computing, offering a new paradigm for neural network design. In this paper, we present SechKAN, a novel KAN based on hyperbolic secant (sech) functions. The hyperbolic secant basis is adopted for its smooth bell‑shaped form, localized responses, and well‑behaved gradients. We employ a 1D linear projection to reduce the number of parameters, allowing SechKAN to maintain a model size comparable to that of multilayer perceptrons (MLPs). Experimental results show the effectiveness of SechKAN on function fitting, PDE surrogate modeling, and image classification benchmarks, including MNIST, Fashion‑MNIST, CIFAR‑10, and CIFAR‑100. On function fitting, SechKAN achieves performance comparable to both MLPs and representative KAN variants. On PDE surrogate modeling, it outperforms MLPs and achieves competitive or better performance than representative KAN variants. On image classification benchmarks, SechKAN achieves the best performance among the evaluated KAN variants while remaining competitive with MLPs using a comparable number of parameters. However, SechKAN still incurs higher computational cost than MLPs and some KAN variants. Our source code is publicly available at https://github.com/hoangthangta/All‑KAN.
Authors:Philip-Roman Adam, Stefanie Schmidtner
Abstract:
Transit signal priority (TSP) requires balancing competing objectives: reducing bus delay while limiting adverse impacts on non‑bus traffic and avoiding extreme waits for a subset of vehicles. Existing reinforcement‑learning (RL) approaches to TSP typically encode transit‑aware features (e.g., occupancy and schedule deviation) but optimize a fixed reward or fixed scalarization, which limits operational flexibility when agency priorities change across time‑of‑day or disruption conditions. We present a preference‑conditioned TSP controller, π(a \mid s,w), that selects the next signal phase under minimum/maximum green and transition‑feasibility constraints and can be tuned at runtime via a preference parameter w to trade off bus‑priority emphasis against overall traffic delay without retraining. We implement this on top of IntersectionZoo by introducing a constrained signal‑control/TSP wrapper, and we extend scenario generation with bus‑prevalence augmentation and timetable‑based bus insertion to address sparse transit‑priority events during training. Experiments against fixed‑time control, a rule‑based TSP overlay, and fixed‑weight PPO specialists show that a single learned conditioned policy spans a smooth empirical trade‑off frontier across runtime preferences, outperforms fixed‑time and rule‑based baselines, and maintains constraint feasibility, while tail‑delay diagnostics reveal that non‑bus externalities remain limited for moderate preference settings but can increase substantially under high bus‑priority weights. The source code of this work is available at https://github.com/urbanAIthi/morl‑tsp.
Authors:Zhifan Song, Haralampos-G. Stratigopoulos, Hassan Aboushady
Abstract:
We present E‑SpecFormer (Edge Spectrum monitoring Transformer) for end‑to‑end automatic modulation and covert channel (CC) recognition. We introduce LiTAN (Linear Tanh Attention Network), a Softmax‑ and LayerNorm‑free attention mechanism that reduces complexity while increasing accuracy in RF tasks. E‑SpecFormer is parameterized in four scalable variants (Nano, Small, Medium, Large) to accommodate diverse hardware constraints. Using the RadioML2018 dataset for modulation recognition, the Nano variant achieves 86.5% average accuracy for Signal‑to‑Noise Ratios (SNRs)>0 dB, and on the hardware Trojan (HT)‑based CC dataset it reaches 94.2% accuracy, both with fewer than 10k parameters and up to speed of 92 μs per frame on FPGA/CPU co‑execution, surpassing state‑of‑the‑art edge models at a fraction of their cost. These results establish E‑SpecFormer as an edge‑efficient solution for real‑time spectrum intelligence on Internet of Things (IoT) devices. GitHub link to the repository: https://github.com/zsniko/E‑SpecFormer.
Authors:Chao Han, Haozhe Hu, Xiaoyu Shen
Abstract:
Large language models (LLMs) are often compressed through static parameter pruning or dynamic token‑level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary. This work asks \emphwhether combining these two mechanisms can delay such degradation by distributing the compression burden. We study a minimalist compound sparsity framework that first applies low‑rank approximation and channel pruning to obtain a statically compressed backbone, and then introduces lightweight routers for per‑token dynamic layer skipping. This design enables independent control of parameter sparsity and token‑level computation sparsity. Experiments across language understanding and modeling benchmarks show that compound sparsity consistently outperforms single‑mechanism compression under the same total sparsity, delaying the decay point on understanding tasks and preserving stronger modeling performance. Further analysis reveals cross‑dimensional interference between parameter pruning and token skipping, and shows that near‑balanced allocation is most effective under a fixed sparsity budget. These results demonstrate that compound compression provides a practical way to improve LLM compression, while revealing a broader cross‑dimensional sparsity boundary that ultimately limits further compression. Code will be available at https://github.com/EIT‑NLP/LLM‑Pruning.
Authors:Plawan Kumar Rath
Abstract:
Multi‑Level Intermediate Representation (MLIR) underlies modern ML compiler infrastructure (TensorFlow, JAX/StableHLO, PyTorch Inductor, IREE), yet appears only in trace amounts in code‑LM pretraining corpora. MLIR is also extensible by design: new dialects ship per application domain, so a fine‑tuned model per dialect does not scale. We ask whether inference‑time priors derived mechanically from each dialect's Operation Definition Specification (ODS) can substitute for gradient‑based adaptation. First, we release four natural‑language‑to‑MLIR benchmarks across three dialects ‑ MLIR‑Spec‑150, Linalg‑Spec‑30, StableHLO‑Spec‑30, and StableHLO‑Held‑Out‑200 ‑ totaling 410 in‑scope NL‑to‑MLIR pairs, plus a 25‑program out‑of‑grammar stress set and a hand‑authored n=30 functional reference set, shipped under Apache‑2.0 with Gebru datasheets and Croissant 1.0 metadata. Second, we build a three‑layer schema‑derived constraint stack: a CFG over op signatures(C1), type‑domain splits from an ODS‑extracted type lattice (C2), and an SSA‑scope validator driving five‑retry rejection sampling (C3). Porting from arith+func+memref+linalg to StableHLO required no new constraint‑layer code. On dialects whose verifier semantics are dominated by structural constraints, schema‑derived priors let SmolLM2‑1.7B match or exceed 15B‑34B code LMs at 8‑25x the per‑generation speed: on linalg, SmolLM2 reaches 80.0% verify‑valid (three‑seed mean, n=125), beating CodeLlama‑34B, Granite‑Code‑34B, and StarCoder2‑15B by 21‑44 percentage points with non‑overlapping CIs. On arith+func and on the templated parametric StableHLO‑Held‑Out‑200, where verifier semantics turn on attribute values rather than structure, the same baselines match or beat the SLM; we scope these as non‑win cells. We release benchmarks, decoder, all per‑prompt generations, and a reproducibility Docker image.
Authors:Ali Toygar Abak
Abstract:
We present Phionyx, a deterministic AI runtime architecture derived from the broader Echoism interaction framework that introduces a governance‑first approach to AI engineering: treating large language model (LLM) outputs as noisy sensor measurements rather than direct decisions. Unlike probabilistic agents, Phionyx enforces deterministic state evolution via a structured state vector governed by deterministic state‑evolution equations, enabling reproducible behavior in applications requiring auditability and governance. The architecture integrates three layers: (1) a deterministic evaluation kernel processing noisy sensor measurements through a canonical 46‑block pipeline, (2) a unified safety layer providing pre‑response control and architectural privacy enforcement, and (3) a semantic time‑based memory system implementing impact‑weighted cache eviction. Experimental validation on single‑instance deployments demonstrates approximately 31% reduction in computational overhead vs. post‑hoc filtering (at 30% unsafe input ratio, simulated cost model) and up to 24% improvement in high‑value data retention vs. LRU (72% vs. FIFO, same cache capacity, benchmark‑verified), deterministic execution verified across 100 repeated runs with zero variance in control signals (hash‑verified), and zero unplanned restarts in single‑instance deployment testing (see Appendix C for methodology and scope). This paper presents the architecture, its analytic structure, and scoped experimental evidence; generalization to distributed or multi‑tenant deployments remains future work.
Authors:Yiyang Cai, Nan Chen, Rongchang Xie, Junwen Pan, Chunyang Jiang, Cheng Chen, Wen Zhou, Zhenbang Sun, Wei Xue, Wenhan Luo, Yike Guo
Abstract:
Human‑object centric video personalization (HOCVP) is a core task within subject‑driven video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter‑subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra‑subject references (e.g., OCR maps, multi‑view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter‑ and intra‑subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference‑level relationships without compromising the controllability of text encoders or incurring costly re‑alignment. Specifically, we introduce global multimodal guidance within self‑attention to better align MLLM‑derived semantic features with VAE tokens. Furthermore, we propose modality‑reference embedding to differentiate tokens from MLLM features and VAE tokens and associate intra‑subject reference image tokens. Extensive experiments validate that our method achieves state‑of‑the‑art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie‑page.github.io/
Authors:Shyamal Y. Dharia, Stephen D. Smith, Camilo E. Valderrama
Abstract:
Real‑time EEG classification on edge devices is bottlenecked by the floating‑point arithmetic of conventional neural networks. We investigated Differentiable Logic Gate Networks (Diff‑Logic) as a hardware‑native alternative that compiles models into pure Boolean circuits executable via bitwise CPU operations. Through rigorous iso‑parameter experiments across four EEG datasets spanning two classification tasks, binary dementia detection and 3‑class emotion recognition, we compared Diff‑Logic against matched‑capacity Multi‑Layer Perceptron (MLP) and Binarized Neural Network (BNN) baselines at four complexity tiers (50k‑500k parameters). On dementia screening, Diff‑Logic achieved 80.2% Macro F1, outperforming the MLP baseline by 6.8%. On emotion recognition, the MLP retained a moderate performance advantage but incurred a 2.3× higher latency and 14× larger model size when deployed on a power‑constrained (7W) Nvidia Jetson Orin Nano CPU (Single‑core). Critically, Diff‑Logic inference time remained nearly constant across a 10× increase in model scale, achieving a peak speedup of 2.9× over MLPs at the largest complexity tier. Our results establish logic‑based neural architectures as a practical paradigm for resource‑constrained brain‑computer interfaces, achieving competitive or superior performance while natively satisfying the latency and memory constraints of portable edge deployment. Code is available on GitHub: https://github.com/Shyamal‑Dharia/eeg‑difflogic
Authors:Wenbo Wei, Jun Wang, Shan Raza, Abhir Bhalerao
Abstract:
Panoptic segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose Position Embedding Modulation with Occlusion Level Attention (PEMOLA), a novel occlusion‑aware module that can be seamlessly integrated into transformer‑based panoptic segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO‑OLAC dataset. The classifier derives the occlusion‑level attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable embedding to produce channel‑wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer Vision Tasks (Cityscapes‑OLAC), following the same labelling protocol as COCO‑OLAC, to evaluate the cross‑dataset generalisation ability of PEMOLA. Extensive experiments on COCO‑OLAC and Cityscapes‑OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion‑level attention helps deliver robust panoptic segmentation under occlusion. Code and dataset are available at https://github.com/wenbo‑wei/PEMOLA.
Authors:Peiyu Zang, Bosen Xie, Ruoxiang Xu, Yongqiang Cai
Abstract:
Physics‑Informed Neural Networks (PINNs) solve PDEs by incorporating physical constraints into neural‑network training, but large‑scale problems are limited by automatic‑differentiation memory overhead and inefficient execution of grid‑based PDE operators. We present FlashPDE, a drop‑in fused operator library for grid‑based scientific machine learning. FlashPDE replaces fragmented PyTorch finite‑difference execution with differentiable Triton kernels. Each operator integrates fused stencil evaluation, an analytic discrete‑adjoint backward pass, and boundary‑gradient correction within a unified PyTorch autograd Function interface. The library provides 14 differentiable PDE operators covering 17 configurations across 1D‑‑3D elliptic, parabolic, and Navier‑‑Stokes systems, while remaining independent of neural architectures and training strategies. Experiments on an NVIDIA A100 GPU show that FlashPDE reduces peak memory usage by up to 37.0x compared with coordinate‑based automatic differentiation and reduces CUDA kernel launches by up to 3.5x compared with eager PyTorch finite‑difference implementations. Across six representative PDE benchmarks, FlashPDE achieves up to 2.30x end‑to‑end time‑to‑solution speedup and up to 19.2x kernel‑level acceleration while maintaining numerical agreement with PyTorch finite‑difference references. FlashPDE provides a hardware‑efficient execution layer that bridges differentiable PDE solvers and GPU‑optimized numerical computation within the PyTorch ecosystem.
Authors:Kehan Li, Bohan Hou, Minghao Zhu, Tianyi Zhang, Zesen Cheng, Zhikai Wang, Sicong Leng, Xin Li, Xiao Lin, Biying Yao, Minghua Zeng, Jiangpin Liu, Ronghao Dang, Jiayan Guo, Siteng Huang, Haoyu Zhao, Heng Ping, Yaxi Zhao, Tong Zhao, Kexiang Wang, Tong Lu, Shengke Xue, Jiahao Tang, Yulei Wang, Zejing Wang, Jianwei Gao, Shijian Lu, Chengju Liu, Jianfei Yang, Mingxiu Chen, Deli Zhao
Abstract:
We present RynnBrain 1.1, a family of embodied foundation models spanning 2B, 9B, and 122B‑A10B scales. Trained with a unified spatio‑temporal and physically grounded framework, RynnBrain 1.1 supports embodied perception, spatial reasoning, localization, and planning. Compared with RynnBrain 1.0, it further introduces contact‑point prediction across the model family and native 3D grounding for the 2B and 9B models, yielding representations and outputs that are more directly aligned with robot manipulation. We also develop RynnBrain‑VLA with a unified cross‑embodiment action space and embodiment‑specific masking, and deploy it on Unitree G1, Astribot‑S1, and Tianji‑Wuji. RynnBrain 1.1 achieves strong results on embodied cognition, localization, and 3D grounding, with the 122B‑A10B model outperforming all evaluated proprietary and open‑source models on VSI‑Bench, MMSI, and RefSpatial‑Bench. Real‑robot experiments show that RynnBrain‑initialized policies outperform Qwen‑based and representative generalist VLAs, while joint multi‑task and multi‑embodiment training improves process scores and success rates over per‑task training.
Authors:Kento Kawaharazuka, Yoshiki Obinata, Hirokazu Ishida, Jihoon Oh, Temma Suzuki, Shintaro Inoue, Keita Yoneda, Ayumu Iwata, Kei Okada
Abstract:
The global competition for developing robotic foundation models is intensifying. Among the data collection systems used for dual‑arm robots, ALOHA is representative of being low‑cost and open‑source, and is widely adopted by researchers as a de facto standard. However, due to its limited ability to generate high forces and speeds, it is difficult to handle heavy objects or perform fast manipulations. To address this, we developed MEVION, a low‑cost and open‑source dual‑arm robot data collection system capable of generating greater force and speed. All parts of this robot can be sourced through e‑commerce, and by extensively utilizing sheet metal welding, its large body structure is constructed with a small number of components at low cost, while also simplifying assembly. MEVION is equipped with four 6‑DoF arms with parallel grippers. Each arm weighs 7.0 kg and has a maximum torque of 60 Nm, and the entire system can be constructed for about USD 14,000. The elbow joint adopts a closed‑link mechanism similar to those used in quadruped robots, which reduces the distal mass and enables higher force and speed output at the end‑effector. We demonstrate that MEVION enables data collection for object manipulation tasks not previously possible and supports imitation learning‑based motion generation. All hardware and software of this work are included in the Supplementary Materials or https://github.com/haraduka/mevion.
Authors:Lingyu Kong, Ruicheng Li, Ruicheng Wang, Sicheng Xu, Chengtang Yao, Jianfeng Xiang, Jiaolong Yang
Abstract:
Monocular geometry estimation has recently achieved impressive performance across diverse scenes. However, state‑of‑the‑art models still face notable distortion in local 3D structure, especially in fine details, like thin structures and small objects. We attribute this limitation to an architectural mismatch: most current models decode 3D geometry within a 2D parameterization, where feature interactions are governed by image‑plane proximity rather than true 3D spatial relationships. This inadvertently mixes features from geometrically distant surfaces, resulting in over‑smoothed geometry particularly around thin or elongated structure. In this paper, we propose MoGe‑3, a fine‑detail monocular geometry estimation model with Self‑Guided Sparse 3D Refinement (SSR) that lifts monocular geometry modeling from 2D image space to 3D space for high‑fidelity metric‑scale point maps. MoGe‑3 lifts the coarse point map from a foundation base model onto a sparse voxel shell and refines it via SSR. The SSR employs sparse convolutions that aggregate features based on 3D spatial locality, avoiding feature mixing across depth discontinuities. Extensive experiments on diverse datasets demonstrate that MoGe‑3 significantly outperforms existing approaches in recovering fine detailed 3D geometry across both quantitative metrics and qualitative visualizations. Project page: https://qft‑333.github.io/moge3page/
Authors:Thu Hang Khuat, Duy-Nam Bui, Thuy Ngan Duong, Manh Duong Phung
Abstract:
In search and rescue operations, there is a period known as the "golden time" during which the probability of finding the target alive is highest. The objective of this work is to propose a new search algorithm for unmanned aerial vehicles (UAVs) with a focus on improving the detection probability and execution time. We approach this problem by first modeling target dynamics as a Markov process and the detection likelihood as a function of image quality and the observer's vision. We then employ Bayesian theory to derive a fitness function representing the probability distribution of the target's location over the search area. Finally, we introduce a new algorithm named polar coordinate‑based differential evolution (PDE) to generate a UAV search path that maximizes this fitness function. The PDE algorithm utilizes polar coordinates to incorporate kinematic constraints and maneuver properties of the UAV, allowing for better exploration of the solution space. A series of simulations and comparative analyses have been conducted to evaluate the performance of the proposed algorithm. Experiments involving a real UAV have also been conducted. Results demonstrate that the PDE algorithm outperforms state‑of‑the‑art algorithms in terms of detection probability and execution time across diverse search scenarios while remaining practical for real‑world applications. The source code of the algorithm is available at https://github.com/thuhangkhuat/PDE_target_search.
Authors:Jiacheng Ding, Cong Guo, Jason Xu
Abstract:
We introduce WC2026‑Agents, a benchmark and dataset for evaluating large language models (LLMs) as autonomous forecasting agents on real, future events. For every one of the 104 matches of the 2026 FIFA World Cup, four frontier models ‑‑ Claude Opus 4.8, ChatGPT (GPT‑5.5, high reasoning), Gemini 3.1 Pro, and Grok (Expert Mode) ‑‑ ran an identical search‑act‑reflect loop: gather evidence with a web tool, commit to a 1X2 (team‑A win / draw / team‑B win) distribution and a virtual 100‑USD bet, and, after the match, reflect given only the final score. Because every match kicked off after the models' training cutoffs, the benchmark is contamination‑free by construction. Crucially, we pair the four agents with a fifth competitor drawn from the same information environment ‑‑ the pre‑match betting market ‑‑ collected as per‑match 1X2 odds, giving an economically grounded baseline and letting us score not just what an agent predicts but what it does with money. The release contains 416 forecasts and 414 reflections with verbatim reasoning, ground truth (including penalty shootouts), odds, and a reproducible evaluation suite. A reference evaluation surfaces findings that raw accuracy hides: the four agents issue an identical top pick in 92% of matches and none beats the market's Brier score; indeed, a naive flat stake on the market favorite out‑earns all four agents. Yet the agents diverge sharply as decision‑makers: betting return‑on‑investment ranges from ‑18% to +10%, fading the market is unprofitable for all four, the share of forecasts that cite the market ranges from 12% to 100%, and self‑reported error rates on wrong picks range from 36% to 86%. The benchmark thus measures calibration, decision quality, and self‑knowledge ‑‑ axes on which frontier models differ even when their predictions do not. Data and code: https://github.com/graphuofm/FIFA2026LLM
Authors:Jun Xue, Zhuolin Yi, Yanzhen Ren, Yihuan Huang, Jiayu Xiong, Yi Chai, Guanxiang Feng, Jiajun Liu, Tong Zhang
Abstract:
Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to controlled additive noise scenarios, lacking systematic investigation of the complex distortions introduced by acoustic front‑end (AFE) processing pipelines in real‑world deployments. In this work, we simulate a unified AFE pipeline comprising acoustic echo cancellation, noise suppression, automatic gain control, and voice activity detection (VAD), and conduct a comprehensive evaluation of current state‑of‑the‑art models. The results show that the nonlinear and time‑frequency coupled distortions introduced by AFE significantly degrade detection performance. To address this issue, we propose a Time‑Frequency Consistency Learning (TFCL) framework, which aims to learn invariant spoofing representations that remain stable before and after AFE processing. We observe that AFE not only introduces temporal misalignment (e.g., segment‑level shifts caused by VAD), but also weakens or distorts critical frequency‑domain cues. To this end, TFCL employs an attention‑driven soft alignment mechanism to capture cross‑temporal dependencies, along with frequency‑domain structural consistency constraints to enforce feature invariance. As a result, the model is able to maintain stable representations under both temporal perturbations and spectral distortions. Extensive experimental results demonstrate that the proposed method effectively mitigates the performance degradation caused by AFE processing, significantly improving the robustness of SDD in real‑world scenarios. The code is available at https://github.com/JunXue‑tech/TFCL.
Authors:Tsubasa Konno, Takahiro Ninomiya, Yukun Zhou, Koichi Ito, Siegfried K. Wagner, Yiqun Lin, Pearse A. Keane, Toru Nakazawa, Takafumi Aoki
Abstract:
Volumetric segmentation of optical coherence tomography (OCT) images is essential for diagnosing ocular diseases but requires labor‑intensive voxel‑wise annotations. While semi‑supervised learning (SSL) can reduce annotation costs, most existing methods process data slice by slice and fail to exploit the inherent 3D spatial context. We propose PC‑Seg, a progressive cross‑view consistency framework that learns high‑accuracy 3D segmentation models from sparse 2D annotations. Unlike conventional multi‑view approaches, PC‑Seg uses a single 2D model to learn cross‑view consistency from standard B‑scans and orthogonal slices, thereby generating reliable volumetric pseudo‑labels. These pseudo‑labels are then distilled into a 3D model, followed by a co‑training stage in which the 2D and 3D models mutually refine each other through ensemble pseudo‑labeling. Experiments on the MSHC and Duke DME datasets demonstrate that PC‑Seg achieves accuracy comparable to fully supervised learning while using labels for only about 0.7% of the training data, outperforming state‑of‑the‑art semi‑supervised and retinal layer segmentation methods. Our code is publicly available at https://github.com/gsisaoki/pc‑seg‑official.
Authors:Lingrui Li, Nan Pu, Dong Zhao, Wenjing Li, Andrew P French, Zhun Zhong, Xin Chen
Abstract:
Test‑time adaptation (TTA) aims to mitigate distribution shifts by adapting models with unlabeled target data at inference time. While TTA with vision‑language models (VLMs) has shown promising results in classification, extending it to medical image segmentation remains challenging. In this setting, the adaptation gains from optimizing on VLM‑generated predictions are often outweighed by the degradation to the VLM's strong pretrained features caused by noisy, update‑driven learning, resulting in limited and unstable improvements. We therefore propose Memory‑Supported Synergistic Adaptation (MSSA), a novel training‑free TTA framework for medical image segmentation. Without updating model parameters, MSSA dynamically selects reliable image‑text predictions to construct an online memory, uses them as text‑guided semantic priors, and couples them with cross‑image structural alignment for robust adaptation. Specifically, MSSA consists of (i) a noise‑aware memory construction module that filters and stabilizes cross‑modal predictions, and (ii) a relevance‑driven prototype alignment module that aligns the target sample with structurally consistent memory samples and their reliable predictions to improve adaptation. Extensive experiments on multiple medical segmentation benchmarks demonstrate that MSSA consistently improves VLM‑based segmentation models and outperforms existing fine‑tuning‑based TTA methods by a clear margin, with gains of up to 12.2% DSC and 11.7% mIoU. Project page: https://lingrayy.github.io/MSSA/ .
Authors:Mansur Arief, Nur Ahmad Khatim, Ali Akarma, Ahmad Alfan Alfian Irfan
Abstract:
Modern software teams have mature tools for low‑level testing, such as pytest, JUnit, and Jest, which make it inexpensive to write unit tests and run them on every commit. Systems engineering, in parallel, has developed rigorous principles for design verification and validation (V&V), which has worked very well across engineering discipline to align user expecations and requirements with developers' deliverables. In practice, however, the two rarely connect, and the link between users' high‑level requirements and the low‑level tests that machines actually run is maintained by hand, if at all. This gap is increasingly costly for AI‑enabled and cyber‑physical systems, for which regulators now ask for traceable evidence that high‑level requirements are met, while raw test results provide little of the structure such evidence requires. We introduce VNVSpec, an open‑source framework that makes V&V specifications machine‑readable and executable. With this framework, users state high‑level requirements directly or import them from catalogs derived from published standards. Then, the framework checks requirement quality, supports decomposition into module‑level requirements with explicit metrics and acceptance criteria, links these requirements to test results through a traceability graph, and compiles the collected evidence into verdicts and audit‑ready reports. We evaluate the framework by self‑application, in which it is continuously assessed in CI against its own specification of 36 requirements verified by 449 tests, completed within limited time which scales linearly and thus can handle up to 10,000 requirements. We also discuss how the framework extends to testing black‑box AI models and AI coding agents. The framework, its full test suite, the catalogs, and the benchmark scripts are available at https://github.com/ai‑vnv/vnvspec.
Authors:Su Guo, Guangce Liu, Haosen Yang, Jiepeng Wang, Cong Liu, Junqi Liu, Haibin Huang, Hongxun Yao, Chi Zhang, Xuelong Li
Abstract:
Current video generation models achieve impressive results in single‑shot generation, yet remain limited in cinematic video generation, where coherent narratives and effective multi‑shot composition require explicit shot planning. To address this challenge, we propose ShotPlan, a framework for explicit multi‑shot cinematic video generation built upon a video diffusion foundation model. Our method introduces learnable planning tokens that capture shot‑level transition cues and can be seamlessly integrated with the original video generation tokens to control transition timestamps. Unlike standard video generation tokens, the proposed planning tokens are equipped with Fractional Temporal Rotary Position Embedding (FRoPE), enabling shot transitions to be modeled at the frame level. Experiments demonstrate that ShotPlan significantly outperforms existing cinematic video generation methods, offering more flexible shot management and stronger inter‑shot consistency.
Authors:Jing Li, Pan Liu, Meng Zhao, Wanli Xue, Yanhong Yang, Xu Cheng, Fan Shi, Jianhua Zhang, Qinghua Hu, Shengyong Chen
Abstract:
Source‑free universal domain adaptation (SF‑UniDA) adapts a pre‑trained source model to an unlabeled target domain under both covariate and label shifts, without access to source data. However, existing SF‑UniDA methods rely on inefficient techniques such as threshold tuning and clustering. Foundation models (FMs), known for their generalization and zero‑shot capabilities, remain underexplored in SF‑UniDA. In this paper, we propose a framework that leverages foundation models (LFM) for SF‑UniDA. We use a vision‑language model (VLM) to compute similarities between target samples and text labels, including those for unknown classes generated by prompting a large language model. The label shift type is determined by analyzing the coefficient of variation of a similarity‑based sample‑level score. Unknown samples are identified using a binary Gaussian mixture model fitted to another similarity‑based metric. Under a consensus strategy, the pseudo‑labels generated by the VLM are refined by the target model initialized with the pre‑trained source model, integrating knowledge from both the source domain and foundation models. Finally, these refined pseudo‑labels are used to train the target model. Extensive experiments across all possible label shifts and multiple benchmarks demonstrate the effectiveness and superiority of our proposed LFM framework. Our code is available at https://github.com/iamjingli/LFM.
Authors:Guanghu Xie, Mingxu Li, Shuo Zhang, Yonglong Zhang, Yifan Yang, Yang Liu, Zongwu Xie, Baoshi Cao
Abstract:
Flow policies can represent multimodal action distributions for robot manipulation, yet a robot must execute one action at each control step. When several proposals are sampled, critic‑based ranking makes data collection depend on value estimates over candidate actions that may be weakly represented in replay. We introduce HCPG‑Flow, an analytic rollout‑time selector that augments SAC‑Flow with hierarchical, object‑centric contact‑progress guidance while preserving its actor and critic objectives. HCPG switches from end‑effector approach to task progress after contact, scores each proposal by the first‑order reduction of a task‑relevant distance, standardizes scores within the candidate set, and executes a temperature‑controlled action embedding. Across ten simulated tasks, HCPG improves mean success over SAC‑Flow on both benchmarks, including a 9.5 percentage‑point gain on Maniskill. Four physical tasks further show high success with a 17.4% reduction in successful completion steps.Project page: https://hitxraz.github.io/HCPG‑Flow/
Authors:Joey Páolo Kardolus, Daan Hendriks, Jaap Jansen
Abstract:
Quantitative joint angles are rarely available in routine care because the tools are slow, costly, or confined to a laboratory. We show that clinical joint angles can be read directly from the per‑segment rotation matrices a parametric body model already produces, with no inverse‑kinematics or musculoskeletal‑model fitting step. On the OpenCap LabValidation cohort, using the GEM‑X body‑model estimator on single‑smartphone video, our pooled mean absolute error is 4.50 degrees over the fifteen joint angles that match the OpenCap Monocular reference set, the same accuracy range as OpenCap Monocular's 4.8 degrees on the same cohort and reference standard, from a much simpler pipeline. The step that connects a body model to clinical angles is a small calibration table rather than an optimisation, so the same procedure transfers unchanged to other body models: repeating it on SAM 3D Body, changing only the table, gives 4.66 degrees, statistically indistinguishable from GEM‑X, and runs in real time from a live single‑camera stream. The method needs no per‑recording inputs beyond the video itself: no participant height, no camera‑intrinsics database, no per‑subject model scaling. This broadens where movement analysis is practical, from in‑clinic and at‑home recording to telerehabilitation and large‑scale decentralised studies.
Authors:Damien Teney, Liangze Jiang, Hemanth Saratchandran, Simon Lucey
Abstract:
Transformers are remarkably versatile and their design is largely consistent across a variety of applications. But are they optimal for any given task or dataset? The answer may be key for pushing AI beyond merely scaling current designs. Method. We present a method to optimize a transformer architecture for a given dataset, which we use as a tool to study optimal task‑specific inductive biases. This method replaces the most important non‑linearities (GeLUs,;softmax) with functions learned on held‑out data. We then train the resulting architectures on other datasets, as a way to evaluate the compatibility between pairs of tasks. Findings. On algorithmic toy tasks, we identify new architectures with dramatic improvements in learning speed, in‑ and out‑of‑distribution generalization, and stability across seeds. The new designs prove very task‑specific however, and indicate that these tasks require inductive biases very different from those of standard transformers. On code and language modeling datasets, we also find architectures with consistent, yet smaller improvements. These designs transfer much better across datasets and domains (English & computer code). Implications. Our results show that standard transformers are rarely a local optimum in the space of architectures. Simple alternatives can perform much better but sacrifice universality. This suggests that there may be room for improved architectures that better support multiple capabilities simultaneously, such as fluency and robust reasoning.
Authors:Yongchan Hong, Defu Cao, Wenjin Liu, Thomas Ku, Jordy Homing Lam, Emily Nguyen, Willie Neiswanger, Vsevolod Katritch, Yan Liu
Abstract:
Accurate protein‑ligand binding affinity prediction is central to computational drug discovery, yet modern docking engines frequently disagree without indicating which prediction to trust. Consensus scoring and ensemble methods improve mean accuracy but treat all predictions identically without interpretable confidence measures or uncertainty decomposition, ignoring the chemical context of each protein‑ligand pair. To address this limitation, we introduce RELIABLE‑BA (RELIABiLity‑aware Evidential fusion for Binding Affinity), an evidential framework for multi‑engine binding affinity prediction. Our model comprises three steps: (1) modeling each engine as an evidential expert via Normal‑Inverse‑Gamma distributions, (2) scaling epistemic uncertainty through learned reliability from molecular context while preserving each expert's predictive mean, and (3) fusing experts through closed‑form aggregation that captures both individual uncertainty and inter‑engine disagreement. Experiments on the PDBBind and BDB2020+ benchmarks demonstrate competitive point prediction with substantially improved uncertainty calibration, and additional validation on the SARS‑CoV‑2 Mpro dataset and 5HT2A receptor demonstrates applicability to clinically relevant drug targets. Crucially, these uncertainty estimates enable reliable filtering of protein‑ligand pairs, reducing prediction error by up to 25% when retaining only high‑confidence pairs. To our knowledge, RELIABLE‑BA is the first multi‑engine binding affinity prediction framework to combine evidential fusion with context‑dependent reliability, offering a principled path toward trustworthy AI‑guided drug discovery. Our code is publicly available at https://github.com/yongchand/RELIABLE‑BA.
Authors:Ruiyi Ding, Jie Li, He Kang, Ziyan Liu, Chengru Song, Yuan cheng
Abstract:
Group Relative Policy Optimization (GRPO) is a powerful reinforcement learning algorithm for aligning generative models with human preferences. While successful in large language models~\citeshao2024deepseekmathpushinglimitsmathematical, its extension to diffusion and flow matching models introduces a severe computational bottleneck: gradients must be back‑propagated through the high‑capacity DiT backbone at \emphevery timestep of the sampling trajectory, making high‑resolution text‑to‑image (T2I) training prohibitively expensive. Training‑free DiT inference acceleration methods (e.g., Δ‑DiT, ScalingCache) exploit the fact that DiT hidden states and velocity predictions vary \emphsmoothly and nearly linearly along the trajectory. We ask whether the same linearity can reduce the backward‑pass cost of DiT RL training, and answer affirmatively with JAGG (Jacobian‑Aggregated Group Gradient), which reduces full transformer backward passes from W to 2 per group of W consecutive steps. JAGG approximates intermediate‑step Jacobians via t‑weighted interpolation of the endpoint Jacobians, then aggregates per‑step upstream signals into two composite gradients applied through a single joint backward pass. We prove this interpolation is \emphexact when the velocity is linear in (z,t), and a cosine‑similarity routing rule (\textttjagg\_frac) deploys JAGG only where the assumption holds. Experiments on T2I benchmarks show JAGG delivers ~2× backward speedup with negligible quality degradation. The code for this work can be accessed through https://github.com/SchumiDing/JAGG.
Authors:Jingzhe Fang, Guozhi Xu, Yunfan Cui, Xiaochen Yang, Zhangyu Hua
Abstract:
AI companions are judged not only by single‑turn fluency but by whether they sustain emotional continuity: remembering who the companion is, what the user prefers, and how the relationship has felt. We present ZifaMem, a structured memory system that organizes dialogue into session summaries, episodic memories, and a consolidated user model. Against a deployment‑honest comparator that supplies the full raw dialogue history, and under a fixed LLM‑as‑a‑judge protocol with route audits, structured memory raises pooled four‑backbone emotional‑intelligence scores by 11.4% (95% CI 6.3% to 17.1%), and persona grounding improves on all four backbones (Claude +42% relative). Multi‑turn affect context wins a +39% net preference over a single‑turn snapshot (exploratory), whereas an additional emotion state machine yields no measurable gain on any of five endpoints. Under an identical preregistered protocol, three memory systems (ZifaMem, Mem0, and filtered verbatim retrieval) each improve significantly over raw‑history deployment, and ZifaMem and Mem0 are statistically equivalent within +/‑5 points on the preregistered primary preference endpoint. The ZifaMem SDK, CLI, and portable Agent Skills are open‑sourced at https://github.com/zifacorp/zifamem.
Authors:Alya Almsouti, Lotfi Mecharbat, Noha Aboukhater, Yousef Alabrach, Siddiq Anwar, Andre Kumar, Ibrahim Almakky, Mohammad Yaqub
Abstract:
Lung ultrasound (LUS) is a bedside tool for assessing pulmonary edema in patients at risk due to heart failure or impaired kidney function. However, automated LUS analysis remains challenging because of speckle noise, imaging artifacts, and operator‑dependent acquisition variability. In this work, we present a deep learning framework for multi‑class LUS video classification that explores two components: hierarchy‑aware training, and anatomy‑guided learning. Starting from a strong baseline, we introduce hierarchical training strategies and then introduce pleural line mask supervision to guide model attention toward anatomically relevant regions. We study four clinically relevant classes‑‑healthy, B‑lines, consolidations, and mixed B‑lines with consolidations‑‑using an open‑access dataset of 1,886 videos from 219 patients, evaluated with patient‑level five‑fold cross‑validation. Results show that hierarchy‑aware training improves pathological separation relative to flat classification, while mask‑guided attention supervision achieves the highest mean macro‑F1 of 65.7% and produces more localized attention patterns. Transfer experiments on the external COVID‑BLUeS dataset further show competitive and parameter‑efficient adaptation while preserving pleural‑focused attention behavior. These findings suggest that combining clinically structured objectives with anatomy‑guided supervision is a practical approach to robust, interpretable LUS video analysis. Code and model implementations are available at https://github.com/Alya‑Almsouti/LUS‑video‑classification.
Authors:Jie Hu
Abstract:
Test‑time collaboration, including self‑consistency, best‑of‑N selection, critic models, and verifier pipelines, is often credited with broadly improving LLM reasoning, yet its gains are uneven and sometimes negative. We ask when training‑free collaboration should be expected to help. For a fixed candidate pool, we decompose a selector or verifier's net gain into measurable factors: recoverable mass, verification‑signal coverage, conditional selection quality, and harm to already‑correct outputs. This reframes collaboration as a candidate‑selection problem rather than as an intrinsic property of a multi‑agent topology. Across LiveCodeBench, MATH Level‑5 hard subjects, and GPQA‑Diamond, gains are bounded first by the oracle gap and then by signal fidelity, which we measure directly as candidate‑level agreement between verifier verdicts and official labels. On LiveCodeBench, a public‑test verifier (MCC 0.825) gains +8.14 percentage points (pp) over a first‑sample baseline; a generated‑test verifier (MCC 0.248) improves by +2.70pp and is not statistically distinguishable from an LLM selector, but operates at near‑zero harm versus the selector's 4.69% harm rate. On MATH, a symbolic answer‑equivalence selector beats self‑consistency by +4.67pp, while LLM selectors are negative. On GPQA‑Diamond, recoverable mass is only 3.03% and 87.54% of candidate pools are answer‑identical; a weaker model's pools shrink both further, suggesting that oracle gap is a joint property of task, model, and sampling configuration. Our framework yields a practical pre‑deployment diagnostic: estimate the oracle gap, then measure coverage, signal fidelity, and harm before investing in collaboration.
Authors:Dairui Liu, Zhongyi Lu, Roger Zhe Li, Changhong Jin, Jitao Lu, Xinyang Shao, Bichen Shi, Mete Sertkan, Aghiles Salah, Aonghus Lawlor, Barry Smyth, Tri Kurniawan Wijaya, Ruihai Dong, Xingsheng Guo
Abstract:
Click‑through rate (CTR) and conversion rate (CVR) prediction are fundamental tasks in online advertising, aiming to estimate the likelihood of user interactions based on various features. While personalized attributes such as age and gender can significantly enhance predictive accuracy, their use is increasingly restricted by privacy regulations, thereby limiting available data for both training and inference. To address this challenge, we propose RAMP (Robust Ad Recommendation Under Limited Personalized‑Feature Availability via Masking and Alignment Pathways), which is designed to improve CTR/CVR prediction accuracy when personalized features are not accessible, thus supporting deployment in privacy‑constrained settings.RAMP consists of (i) a personalized pathway built upon a dual‑tower component with identical inputs but independent parameters, where output masking separates predictions for personalized and non‑personalized signals, (ii) a separate non‑personalized pathway trained with non‑personalized features only, and (iii) a distillation‑inspired prediction‑alignment architecture between (i) and (ii) that improves prediction when personalized features are unavailable. We conduct comprehensive experiments using both public benchmarks and industrial datasets to evaluate the performance of RAMP. Our evaluation spans multiple backbone models and different settings: with and without access to personalized features. The results show that RAMP consistently outperforms state‑of‑the‑art methods when personalized features are missing, while maintaining competitive performance when all features are available. %demonstrating its effectiveness and practicality for real‑world advertising systems. Our code is publicly available at https://github.com/Ruixinhua/RAMP.
Authors:Siobhan Reid, Zhixiang Chi, Li Gu, Omid Reza Heidari, Ziqiang Wang, Yang Wang
Abstract:
Few‑shot Test‑Time Domain Adaptation (FSTT‑DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples. This setting is more realistic than typical domain adaptation setups, which assume access to target data during source training. However, prior FSTT‑DA approaches fail to effectively leverage source domain‑specific knowledge, relying on shallow batch normalization updates, prompt‑based methods that treat the model as a black box, or ensembling strategies that do not capture cross‑domain relationships. To address these limitations, we introduce a new FSTT‑DA framework that integrates LoRA fine‑tuning with model merging. In our approach, separate LoRA modules are fine‑tuned on CLIP's vision encoder for each source domain. Since LoRA modifies only a small fraction of the model's parameters, it retains the base model's generalized knowledge while internally learning domain‑specific features. To adapt the learned knowledge to a specific target domain, we propose a hypernetwork trained via meta‑learning that generates per‑column merging factors to combine LoRA modules. Given a small batch of target images, the hypernetwork produces merging weights that fuse source LoRA modules into a single adapted representation. Our results demonstrate state‑of‑the‑art performance across various domain adaptation datasets. Our code is publicly available at https://github.com/nahbois4321/DA‑MergeLoRA.
Authors:Ricardo Job, Andre Hora
Abstract:
A platform‑specific API is implemented for a particular platform (e.g., operating system), thus, it may not work on other platforms than the target one. Detecting the usage of such APIs is important for supporting software maintenance, as it allows maintainers to be alerted about APIs that could pose potential risks. This paper proposes PSASpotter, a tool to detect the usage of platform‑specific APIs in Python systems. PSASpotter also identifies whether the platform‑specific APIs are used within a defensive code, such as try/except blocks or if blocks that check the current platform. PSASpotter can support the development of novel empirical studies about the usage of platform‑specific APIs in the Python ecosystem. Moreover, the defensive code detected by PSASpotter may contain alternative solutions for unavailable APIs, which can provide insights for software development and testing across multiple platforms. PSASpotter is available at: https://github.com/ricardojob/PSASpotter. Tool video: https://youtu.be/d3WyozTAKS8.
Authors:Yongsen Zheng, Ruilin Xu, Ziliang Chen, Guohua Wang, Mingjie Qian, Jinghui Qin, Liang Lin
Abstract:
The Matthew effect is a notorious issue in Recommender Systems (RSs), \emphi.e., the rich get richer and the poor get poorer, wherein popular items are overexposed while less popular ones are regularly ignored. Most methods examine Matthew effect in static or nearly‑static recommendation scenarios. However, the Matthew effect will be increasingly amplified when the user interacts with the system over time. To address these issues, we propose a novel paradigm, Hypergraph‑Enhanced Multi‑Preference Learning for Alleviating Matthew Effect in Conversational Recommendation (HyCoRec), which aims to alleviate the Matthew effect in conversational recommendation. Concretely, HyCoRec devotes to alleviate the Matthew effect by learning multi‑aspect preferences, \emphi.e., item‑, entity‑, word‑, review‑, and knowledge‑aspect preferences, to effectively generate responses in the conversational task and accurately predict items in the recommendation task when the user chats with the system over time. Extensive experiments conducted on two benchmarks validate that HyCoRec achieves new state‑of‑the‑art performance and the superior of alleviating Matthew effect. Our code is available at https://github.com/zysensmile/HyCoRec.
Authors:Aleksander Fafuła
Abstract:
Abliteration ‑ deleting a model's refusal direction from its weights ‑ is the standard recipe behind popular "uncensored" open‑weight models. We show the surgery is not clean. As a disposition probe we use 21,600 decisions under uncertainty ‑ weekly up/down calls on 60 Warsaw Stock Exchange equities over 18 weeks, replayed through a frozen pipeline so the decision‑layer model is the only variable. The task elicits no refusals at all, so any between‑arm delta is pure side effect. Holding provenance constant (official BF16 checkpoints, a single abliteration author, an identical serving stack, one byte‑identical frozen prompt), we compare base and abliterated arms of two Mixture‑of‑Experts families, Gemma‑4‑26B‑A4B‑it and Qwen3‑30B‑A3B‑Instruct‑2507. Three effects replicate across both families (weeks‑clustered bootstrap CIs excluding zero): abliterated models are systematically more optimistic (+12.2 pp Gemma, +7.4 pp Qwen; the confirmed preregistered endpoint), justify themselves at greater length, and use fewer explicit uncertainty words in forced self‑critiques (both exploratory). A fourth effect reverses sign: the same operation makes Gemma‑abliterated less confident and Qwen‑abliterated more (family CIs non‑overlapping) ‑ one weight surgery, opposite shifts in expressed confidence. Capability covariates rule out instruction‑following degradation as the driver, and no arm shows economic skill: the apparent edge of abliterated arms is regime beta, not alpha. Our provenance audit also caught two independent contamination channels ‑ a mismatched‑quantizer pilot pair and a stale community chat template that silently mangled the rendered prompt ‑ suggesting toolchain artifacts are the rule in studies of community‑modified checkpoints. Whoever deploys an "uncensored" model as an agent is deploying a measurably different decision‑maker, not the base model minus refusals.
Authors:Ayoub Ghriss, Sourav Chakraborty
Abstract:
Linear attention promises constant‑time recurrent inference but degrades sharply on associative recall. We formulate attention recall as a spherical‑packing problem and introduce Kernelized Linear Attention Activations (KATA), a framework whose feature maps are derived from first principles by certifying nonnegative attention weights through a self‑dual homogeneous cone. Building on this observation, we show that rank‑one positive semi‑definite (PSD) features offer a favorable capacity‑‑interference tradeoff. KATA recovers a parameter‑free convex output gate and characterizes associative capacity through the Welch interference floor. For tolerances above this floor, KATA enlarges the state without adding parameters and admits spherical codes with exponentially many keys in the projection dimension. We implement KATA as fused Triton kernels at two operating points: a flash‑attention‑style forward up to ~1.6× FlashAttention‑2 throughput, and an exact O(T) chunked‑state form that reaches ~11× FlashAttention‑2 forward throughput at 131k tokens. An associative scan of the first‑order feature lowers the inter‑chunk recurrence depth to O(\log(T/C)) for chunk size C and averages ~2.4× the throughput of a matched sequential linear‑attention baseline. On long‑range MQAR and repeated‑key overwrite, several KATA variants outperform Gated DeltaNet, with parameter counts and state sizes reported alongside accuracy. Induction preserves near‑perfect recall, while kernel benchmarks show that the maps can be implemented efficiently. KATA retains 0.985 MQAR at a 16× out‑of‑distribution length, approaching the softmax with roughly one quarter of the KV‑cache entries. Experiments on 340M‑parameter LLMs reveal a feature‑dependent fluency trade‑off and clarify how positional embeddings, delta rules, and decay gates interact with feature geometry.
Authors:Ruogu Chen, Weihua Xiao, Ramesh Karri, Jie Han
Abstract:
Analytical placers rely on differentiable objective functions to guide placement, typically combining intermediate surrogate metrics such as half‑perimeter wirelength (HPWL) and cell‑density penalties. However, these placement‑stage surrogates remain misaligned with downstream routed and timing quality. Prior work reduces this gap with human‑designed terms or learned black‑box surrogates, but the former requires expert retuning and the latter is difficult to explain, debug, or deploy in analytical placement flows. CoEvoP&R addresses these limitations with a large language model (LLM)‑based framework that automatically evolves analytical placement objectives. At each generation, the prompt combines the restricted objective interface, baseline context, and archived prior candidates with routing‑related feedback from placement, timing proxy, and routing tools. The LLM proposes readable differentiable objectives, which are embedded and validated in DREAMPlace, evaluated through a timing proxy and an actual router, and stored with their feedback to guide later generations. Across eight ChiP‑Bench Nangate45 designs and three seeds, CoEvoP&R reduces post‑route routed wirelength and congestion by 16.9% and 36.7%, with gains of 0.70 ns in worst negative slack and a 912 ns reduction in total negative slack magnitude over native DREAMPlace. Across eight ICCAD 2015 Superblue designs, it reduces post‑route routed wirelength and congestion by 5.4% and 23.2%. Code is available at https://github.com/FCHXWH823/CoEvoP‑R.git.
Authors:Yuhang Wen, Mengyuan Liu, Zixuan Tang, Junsong Yuan, Sirui Li, Beichen Ding
Abstract:
Understanding physical human‑robot and human‑human interactions is a challenging yet emerging topic in 3D vision. While most existing methods rely on skeleton sequences‑‑effective in low‑light and privacy‑sensitive environment‑‑they face two major challenges: 1) learning and effectively exploiting interaction cues from skeletal data, and 2) compensating for the lack of visual information absent in skeletons alone. To address these challenges, we propose skeletal token alignment and rearrangement (STAR) for human‑robot and human‑human interaction recognition. It learns interaction‑specific skeleton features and enriches them using visual cues by aligning skeleton and RGB video representations in a shared latent space. Specifically, STAR consists of three key components. First, we design a skeleton encoder that captures fine‑grained interdependencies using Entity Rearrangement (ER) and Interactive Spatiotemporal Tokens (ISTs). Second, we present Visual Interaction Encoding that introduces a Focus on Interactions (FoI) strategy to attend to spatiotemporal regions relevant to interactions in RGB videos. Finally, these representations are aligned via a contrastive learning objective, with a refinement head further refines predictions. During training, STAR leverages both skeleton and RGB video data to learn robust, discriminative interaction representations. At inference time, it operates on skeletons alone, retaining visual‑informed benefits while preserving skeleton‑only efficiency. Extensive experiments on Chico, HARPER, NTU Mutual 11 and 26 datasets consistently validate our approach by demonstrating superior performance over state‑of‑the‑art methods. Our code is publicly available at https://github.com/Necolizer/STAR.
Authors:Rongjun Ge, Dongyang Wang, Heng Zhu, Zhirui Li, Yang Chen, Yuting He
Abstract:
Interactive egocentric medical image segmentation (IEMIS) plays an important role in smart‑glasses‑assisted medical image review, segmenting the medical targets a clinician refers to from their egocentric view. Once it succeeds, the object‑level visual evidence it provides strengthens the review and underpins fine‑grained analysis and clinical decision‑making. However, the instruction and the video both come from the user's egocentric perspective, which poses two challenges. (1) Semantic ambiguity leaves the model unable to confirm the user‑intended target. (2) Visual variability makes the segmentation jump from frame to frame. In this paper, we propose EgoMed‑Agent, a multi‑agent system that understands the target from the human perspective through two workflows. (1) The Target Confirmation Workflow grounds the instruction against candidate targets with a reliability score, confirming the target when the grounding is reliable and asking the user to clarify when it is not, thereby confirming the segmentation target. (2) The Localization‑Guided Propagation Workflow couples mask propagation with per‑frame target localization, using the localized target to correct the propagated mask whenever the two diverge, so the segmentation stays on the target across the egocentric video. Extensive experiments show that EgoMed‑Agent reaches 71.34% average Dice, far above the best text‑prompted baseline (11.70%). Our code is available at \hrefhttps://github.com/wdyyyyyy/EgoMed‑Agentour project page.
Authors:Liam Davis, Duo Zhou, Huan Zhang, Guy Katz, Clark Barrett, Haoze Wu
Abstract:
In this work, we investigate the effect of lookahead branching strategies in neural network verification. We present a general recipe to integrate lookahead into any branch‑and‑bound verifier and demonstrate how one of the current state‑of‑the‑art branching heuristics, FSB, can be viewed as a special instantiation of the lookahead branching strategy. We also describe how, in addition to improving the quality of branching decisions, lookahead can generate additional lemmas that accelerate verification. We instantiate the method in two representative branch‑and‑bound‑based verifiers (Marabou and α‑β‑CROWN), and demonstrate that lookahead leads to consistent speedups in verification time and up to 57% more solved instances. Code is available at https://github.com/ai‑ar‑research/lookahead‑branching.
Authors:Param Chordiya
Abstract:
Single‑stream autoregressive decoding of large language models is bound by memory bandwidth: each generated token requires one full forward pass through the target model, and successive passes cannot be parallelized. Speculative decoding restructures this computation: a small draft model proposes K tokens autoregressively, the target model scores all of them in one batched pass, and a rejection‑sampling rule provably preserves the target model's output distribution. We present a from‑scratch, device‑agnostic (CUDA/MPS/CPU) implementation and an empirical study across five draft/target backend configurations on a consumer Apple‑silicon laptop. Distribution equivalence is verified at three levels, culminating in a two‑sample test over roughly 9,200 real‑model tokens per method (χ^2 = 162.5, dof = 200, p = 0.976) and exact greedy‑sequence agreement. The best configuration reaches a measured 1.61× wall‑clock speedup at K=6, on an acceptance profile declining from 69.7% at K=1 to 37.8% at the optimum, while three of five configurations decelerate, either because the draft fails to out‑speed a small target or because the quantized Metal backend executes "parallel" verification serially, an effect we isolate and quantify. The failures are as instructive as the successes: speculative decoding pays off only when verification is genuinely batch‑parallel and the draft/target latency gap is real.
Authors:Dooho Lee, Jaemin Yoo
Abstract:
Node representation learning has advanced rapidly, yet most existing methods rely on per‑dataset training and hyperparameter tuning. This dataset‑specific optimization comes from the difficulty of designing reusable graph models that generalize across diverse graph datasets. In this work, we introduce Node4All, a node representation learner applicable to arbitrary graph datasets without any dataset‑specific optimization. Node4All is built on two complementary ideas. At the architectural level, we introduce the Channel Graph Transformer (CGT), which enables a single fixed parameterization to process arbitrary graph datasets. At the learning level, we propose a self‑supervised learning based on a series of synthetic graphs. Together, these components enable generalization beyond individual datasets, which is infeasible with existing architectures and learning frameworks. We extensively evaluate Node4All on node classification across 25 benchmarks against 21 baselines, covering both supervised and self‑supervised methods. Despite all baselines being trained and optimized for each dataset, a single Node4All, applied uniformly across the datasets, achieves a competitive ranking of 5th among 21 baselines. Moreover, Node4All supports one‑shot and in‑context learning with an appropriate predictor and outperforms recent graph foundation models (GFMs) in these settings. These results demonstrate that Node4All not only achieves reusability across arbitrary graph datasets, but also remains an effective solution in practice. Code and model checkpoints are available in https://github.com/dooho00/node4all.
Authors:Zhanbo Li, Shifeng Wu, Xiangjin Meng, Wenjie Cai
Abstract:
Large language models encode world models implicitly in neural weights, which exposes four structural risks in high‑precision domains such as medicine and finance: hallucination, frozen knowledge, poor explainability, and poor modifiability. This paper proposes data‑first ontology: LLMs are treated as reasoning and language engines, while deterministic knowledge is moved into an explicit multimodal database, DaoQL. We formalize an explicit world model and show that, under rule independence, deterministic evaluation, and fixed conflict resolution, explicit models provide a sufficient condition for composable counterfactual decomposability; implicit models lack atomic read/delta semantics and therefore provide no comparable architectural guarantee. The implemented system focuses on DaoQL's verified storage layer and explicit Eval path, integrating graph, column, vector, and full‑text engines within one process. KVCache graph nodes, expert hot updates, and the DaoQL‑Agent runtime remain future work. On an embedded same‑machine setup, DaoQL reports graph BFS at 1.20 ms, HNSW at 83.1 us, and a Fluent hybrid query at 105.8 us; these results indicate engineering potential but must be interpreted with deployment‑shape differences from client‑server systems. Exploratory measurements on LDBC SNB SF1 and ANN‑Benchmarks further show 34/34 query coverage with interactive‑class queries mostly in the sub‑millisecond to millisecond range, but only 1.8 QPS overall due to long‑tail BI/IC queries; ANN‑Benchmarks reaches Recall@10 >= 99% at thousand‑level QPS after a bridge‑edge protection fix. In a five‑domain counterfactual experiment (n = 1250), DaoQL+GPT‑4o achieves 94% composable counterfactual decomposability, 49 percentage points above GPT‑4o alone. The paper explicitly separates provable structure, preliminary empirical evidence, and architectural roadmap claims.
Authors:Ze Rong
Abstract:
PairUAV relative localization maps two UAV images to a polar navigation command. Although heading and range share the same pairwise pose context, treating them as homogeneous coordinates forces both outputs to use the same decoder evidence and optimization state. Controlled readout probes reveal a different structure: the two axes favor different decoder‑depth combinations, their best checkpoints disagree on 80.8% of a validation trajectory, and range errors exhibit a distinct high‑error tail. We introduce method, Polar Axis‑Conditioned Estimation, which retains a shared Reloc3r‑style pair representation while assigning axis‑specific readout interfaces. Heading uses mid/late relational evidence, whereas range remains attached to a direct late metric path. On the official hidden test, the strongest released raw predictor scores 0.002460; the complementary PAAER predictor scores 0.002514 with a slightly lower angle error. Deterministic challenge packaging, reported separately from learned estimation, yields the final score of 0.001874. Code, checkpoints, predictions, and reconstruction tools are available at https://github.com/zerong7777‑boop/PairUAV‑PACE.
Authors:Peiji Yu, Xin Chen, Tianxing Wu
Abstract:
Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing. However, LLMs often suffer from hallucinations and lack of relevant knowledge when dealing with question answering (QA) tasks. To mitigate these issues, knowledge graphs (KGs) have been utilized to enhance LLM reasoning. Nevertheless, KGs often contain noise and errors, while existing KG‑enhanced LLM approaches are generally unable to identify and filter such noisy and erroneous content, which can instead amplify hallucinations and pose challenges for reliable reasoning. Uncertain knowledge graphs (UKGs), which associate each triple with a confidence score to quantify uncertainty, offer a promising direction to address this challenge. Compared with prior work, we investigate how to leverage UKGs to support LLMs for QA. We propose Debate‑on‑Graph (DoG), a new framework that enables LLMs and UKGs to collaborate adaptively for reliable reasoning. Specifically, we first design a heuristic search algorithm tailored for UKGs to extract reliable and question‑relevant subgraphs, thereby reducing noise and errors in retrieved knowledge. We then introduce a Multi‑Agent Debate mechanism, which yields reliable answers through adaptive adversarial debates, aiming to fully exploit the knowledge in UKGs while preserving the reliability of retrieved evidence. Extensive experiments on four benchmark QA datasets show that DoG achieves state‑of‑the‑art performance over existing LLM reasoning methods and KG‑based baselines, while enabling reliable and adaptive reasoning. Our code is available at https://github.com/seucoin/Debate‑on‑Graph.
Authors:Chen Wang, Zhaochun Li, Jionghao Bai, Yining Zhang, Hexuan Deng, Ge Lan, Yue Wang
Abstract:
Large language model (LLM) post‑training is essential for improving reasoning, adaptation, and alignment. Existing methods mainly follow two paradigms: reinforcement learning (RL) and on‑policy distillation (OPD). However, RL relies on coarse‑grained outcome supervision, resulting in difficult credit assignment and limited capability to acquire new knowledge. OPD, meanwhile, unconditionally matches teacher logits through KL divergence, which creates a dilemma: similar teachers provide little new knowledge, while substantially different teachers often yield ineffective guidance, largely restricting OPD to within‑family distillation. We propose Distilled Reinforcement Learning (Distilled RL), which integrates teacher supervision into the RL objective to provide fine‑grained guidance, selectively transfer new knowledge and avoid unconditional imitation. Distilled RL contains three components: reverse importance sampling with clipping, negative sample reset, and sequence‑level geometric normalization. Through a concise and interpretable case study, we demonstrate that Distilled RL can effectively transfer previously unavailable knowledge from a teacher model to a student model. Extensive experiments across both within‑family and cross‑family distillation settings show that Distilled RL substantially outperforms standard RL and OPD in terms of both pass@1 and pass@k. Our code is available at https://github.com/597358816/Distilled‑RL.
Authors:Mohaimin Al Barat, Hexuan Yu, Shaoyu Li, Yang Xiao, Yi Shi, Eric W. Burger, Y. Thomas Hou, Wenjing Lou
Abstract:
Dynamic Spectrum Sharing (DSS) is a cornerstone of next‑generation wireless systems, yet existing solutions such as Spectrum Access Systems (SAS) rely on centralized administrators that expose sensitive operational metadata and lack cryptographic transaction accountability. Though SAS administrators, such as Google, have introduced pay‑as‑you‑go pricing models, these approaches still face significant privacy and accountability challenges as DSS evolves toward a more open and large‑scale spectrum marketplace. We present SpexPay, a privacy‑preserving and auditable pay‑as‑you‑go spectrum usage framework that enforces fine‑grained, usage‑linked payments without revealing user identities. Spexpay integrates BBS+ verifiable credentials, unlinkable session pseudonyms, and selective‑disclosure proofs to enforce privacy‑preserving access authorization, while leveraging Solidity‑based smart contracts to realize automated and non‑repudiable escrow settlement. By recording only pseudonymous usage evidence and hash‑chained metering data on‑chain, the system achieves strong unlinkability while preserving verifiable accountability and auditability. A full prototype demonstrates low end‑to‑end latency (\approx150 ms) and modest on‑chain cost (\approx603K gas or \approx\0.9), showing that SpexPay is practical for real‑world DSS deployments. We also evaluated the user‑side cryptographic operations on a Raspberry Pi 5 to assess scalability and suitability for edge‑class hardware. Our code and artifacts are publicly available at https://github.com/iambarat/SpexPay.
Authors:Shivanshu Agnihotri, Snehashis Majhi, Deepak Ranjan Nayak, Dwarikanath Mahapatra, Debesh Jha
Abstract:
Automated polyp segmentation in colonoscopy continues to pose challenges due to substantial appearance variations and indistinct polyp boundaries. Although emerging foundation models (FMs) such as DINOv2, SAM, and OneFormer, demonstrate remarkable generalization capabilities, their direct transfer to the polyp segmentation task and deployment in real‑time clinical settings are difficult due to lack of large‑scale labeled data and high computational demands. In addition, adopting multiple FMs together raises concerns, even though they encode complementary semantic and structural information. While lightweight models, including U‑Net, PraNet and U‑Net++, are computationally efficient, they often struggle to generalize across datasets due to limited representational capacity. To address this gap, we propose Lite‑Polyp Inductor (Lite‑Pi), a novel foundation model induction framework that significantly enhances lightweight polyp segmentation baselines. Our proposed framework generates FM‑specific prototype representations and aligns them semantically with the corresponding foundation model priors through reconstruction‑based supervision. Subsequently, transformer‑based fusion is introduced to highlight the polyp relevant representations, including salient boundary information, while preserving complementary semantic cues. Extensive experiments across five polyp segmentation benchmark datasets demonstrate that Lite‑π significantly improves lightweight baselines, achieving superior generalization performance with minimal computational overhead and thereby, offering a practical solution for generalized polyp segmentation. Our code is available at GitHub. https://github.com/lostinrepo/Lite‑Pi
Authors:Haocheng Xia, Yongjoo Park
Abstract:
LLM agents can leak privacy (e.g., paths, emails) and credentials (e.g., API keys) as agent observations (e.g., tool outputs, shell logs, and file reads) are appended to provider‑bound transcripts. Existing placeholder redaction is brittle: it can miss embedded or cross‑turn references, over‑redact benign lookalikes, and destroy the structure useful for reasoning. We present SlotGuard, a local transcript boundary that can hide sensitive data while retaining agents' performance. SlotGuard rewrites structural bindings as typed, suffix‑aware slots, replaces secrets with format‑preserving synthetic values, links cross‑turn references with a lightweight session graph, and restores raw values only inside the trusted runtime. On controlled repository‑oriented agent transcripts, SlotGuard removes all 20,814 annotated structurally sensitive characters across 9,229 paths and reduces credential leakage to 0.0% across 852 planted values. It remains close to raw‑transcript task success across four upstream models, while generic redaction drops to 2.5%. Transcript rewriting takes a median of 14.424~μs per agent turn. The code is publicly accessible at https://github.com/illinoisdata/SlotGuard.
Authors:Feng Xue, Wu Chen, Mingshuai Zhao, Guofeng Zhong, Anlong Ming, Haozhe Wang, Dianqiao Lei, Zhaowen Lin, Haiyang Zhang, Nicu Sebe
Abstract:
Recent geometric foundation models (e.g., Metric3D, Depth Anything and UniDepth) have substantially improved monocular depth estimation (MDE) in both cross‑scene generalization and metric‑scale prediction, yet these gains have not translated to tiny models. We bridge this gap with DepthART (Depth Anything Rethought for Tiny Models), which is a compact MDE model for on‑device deployment across diverse scenes. We first identify two capacity‑driven bottlenecks in tiny models: (i) overfitting to dataset‑specific distribution bias and (ii) unstable metric adaptation under camera shift, where full fine‑tuning easily damages transferable geometry. Accordingly, DepthART combines two simple but effective strategies: a bias‑resistant data sampling scheme to reduce distribution bias under the same training budget, and a camera‑conditioned fine‑tuning protocol that freezes the distilled encoder and adjusts metric scale conditioned on intrinsics while better preserving cross‑dataset generalization. Across datasets, DepthART consistently surpasses previous tiny baselines in both zero‑shot generalization and metric accuracy (e.g., zero‑shot δ_1=0.964 for DepthART‑S on NYUD v2), and in some cases approaches heavy models. We further provide a scalable model family, with DepthART‑S reaching 347/245 FPS (strict FP32) on an RTX A6000 at 224^2/448^2, 102 FPS (TF32) on a Orin NX 8GB, and over 15 FPS (FP32) on a Jetson Nano 4GB.
Authors:Lingwei Dang, Juntong Li, Zonghan Li, Hongwen Zhang, Liang An, Wei Min, Yebin Liu, Qingyao Wu
Abstract:
Hand‑Object Interaction (HOI) synthesis is a cornerstone for animation production and embodied AI. Despite the strong priors of video foundation models, multi‑view consistent HOI synthesis remains challenging due to complex hand motions and occlusions. We present HarmoHOI, a unified diffusion framework that jointly and harmoniously generates synchronized multi‑view HOI videos and globally aligned 3D point tracks. Our core insight is that robust multi‑view consistency fundamentally requires globally aligned 3D geometry and motion. To this end, we propose a Mixture of Multi‑view Diffusion Transformer that co‑models RGB videos and 3D point tracks. By representing point tracks as pseudo‑videos, we align 3D geometric signals with the 2D latent space of foundation models, thereby minimizing the domain gap and easing adaptation of priors. To further ensure geometry consistency, we introduce Global Motion Aligning Diffusion, which refines coarse point tracks into metric‑scale, globally aligned 3D trajectories. HarmoHOI enables on‑the‑fly co‑evolution of 2D appearance and 3D motion during denoising. To overcome the scarcity of multi‑view HOI data, we employ a hybrid data curriculum learning strategy that successfully transfers generic priors from single‑view data to synchronized multi‑view generation. Experimental results show that HarmoHOI achieves state‑of‑the‑art performance in visual quality, motion plausibility, and multi‑view geometric consistency. Project page available at https://droliven.github.io/HarmoHOI_project.
Authors:Shiyuan Piao, Fan Zehui, Yang Liu, Hong Cheng, Juepeng Zheng, Jie Zhou, Fugee Tsung
Abstract:
Accurate short‑term wind power forecasting is essential for grid stability and operational planning, yet remains challenging due to the complex interactions between atmospheric conditions and turbine dynamics. However, existing methods fail to effectively incorporate weather forecasting with wind turbine data (i.e., SCADA), leading to suboptimal solutions. To address this, we introduce a multimodal framework that integrates historical point‑based SCADA data with grid‑based Numerical Weather Prediction (NWP) forecasts, which is challenging due to heterogeneous input and the complex physical wind‑turbine interactions. Our approach first explicitly decomposes inputs into scalar and vector features to better capture both site‑specific and geometric dependencies and then incorporates a geometric encoder to extract rotation‑invariant features from wind vectors. We further leverages a Fourier Neural Operator (FNO) architecture, which performs global convolutions in the frequency domain to efficiently model long‑range spatiotemporal relationships. Extensive experiments on three real‑world wind farms, with weather forecasting data, demonstrate that our model consistently outperforms state‑of‑the‑art baselines, highlighting the effectiveness of its physically‑informed design. The core implementation of our method is publicly available at: https://github.com/shawn‑sypiao/GWPF.
Authors:Afiq Abdillah Effiezal Aswadi, Haotong Ma, Susan Wei
Abstract:
A Bayes‑filtered transformer (BFT) is a transformer trained on sequences that are generated in two steps: first a latent task is drawn from a prior, then observations are drawn conditional on that task. Trained under autoregressive log loss, the BFT's next‑token prediction, in the idealized limit, is the Bayesian posterior predictive distribution (PPD) induced by that prior and that conditional law. In practice the trained BFT is only an approximation of this ideal PPD, raising an interpretive question: what prior and posterior over the latent task has the trained BFT actually internalized? Existing work answers this question by comparing the trained BFT's predictions against the predictions of various "reference" posteriors, each standing in for a different candidate algorithm or computation the BFT might be implementing. This prediction‑space comparison is fragile: different posteriors can share the same posterior‑mean predictions. We use predictive Monte Carlo (PMC) as a general interpretability tool for any BFT: using only next‑token generation, PMC returns an approximation to the implicit prior and posterior over the latent task, answering the interpretive question directly in latent space. We apply PMC to three stylized task families spanning 0‑Markov and 1‑Markov exchangeability. The phenomena previously reported in these settings remain visible in latent space. Code is available at https://github.com/afiq‑aswadi/bft‑pmc
Authors:Tarun Tomar
Abstract:
Vision‑language models normally execute the same complete vision encoder for every question, even when OCR, counting, object, attribute, and spatial queries may not require identical computation. We study whether fixed‑budget combinations of vision blocks can be skipped without fine‑tuning. A shared K‑block route skips one searched set of exactly K blocks for every question, while a capability‑specific K‑block policy selects one same‑size route using a known capability label. We introduce a source‑balanced evolutionary search and compare it with independent ranking, contiguous removal, and random routes at matched budgets. Experiments use Qwen2.5‑VL‑3B‑Instruct, SmolVLM2‑2.2B‑Instruct, and an 876‑example image‑disjoint selection split. Search transfers across architectures: on SmolVLM2, the searched shared four‑block route beats independent construction by 4.91 percentage points. Capability specialization is less stable. On Qwen, the six‑block capability policy beats the shared route by 2.17 points, driven by a 7.10‑point OCR gain. On sealed IIIT5K, however, the SmolVLM2 OCR‑specific route trails its shared route by 13.6 points. Combinatorial search reliably improves route construction, but capability labels do not define universally transferable vision pathways.
Authors:Yaohan Yang, Minglei Shi, Borui Zhang, Jie Zhou, Jiwen Lu
Abstract:
GUI agents must reason about how actions transform interface states, but end‑to‑end success rates entangle this ability with perception, grounding, planning, and recovery. We introduce EvoGUI, a diagnostic framework that converts normalized GUI trajectories into three complementary visual question answering probes: temporal ordering, inverse action/value prediction, and contrastive one‑step successor discrimination. Their labels are derived from trajectory order and logged actions, requiring no additional task‑label annotation after trajectory normalization. We instantiate EvoGUI‑Bench from Mind2Web and WebLINX, yielding 3,000 instances across 120 domains, and evaluate 28 vision‑language model configurations zero‑shot. The strongest model reaches only 60.4 EvoGain, while model scale and GUI specialization do not reliably predict performance. These results establish EvoGUI‑Bench as a scalable diagnostic complement to end‑to‑end GUI‑agent evaluation while exposing substantial headroom in state‑transition understanding. The source code is publicly available at https://github.com/Yyhhh6/EvoGUI.
Authors:Lucky Verma
Abstract:
LLM constraint reasoners are often evaluated near the random‑SAT phase transition, confounding density and solver hardness. We test instance‑level transfer while near‑matching clause density. At aligned size bins, with near‑matched density and matched maximum clause width, we compare proof‑hard expander‑Tseitin and proof‑easy ladder‑Tseitin formulas, pigeonhole anchors, and density‑mismatched controls. Theory separates their resolution hardness; a solver‑specific Glucose mean‑conflict proxy differs by up to 51×, and five other solvers preserve the direction. Across three included models (243 instances each; a fourth is excluded for abstention), the near‑matched‑density accuracy gaps range from ‑32 to +20 points, with a pooled gap of +1.7 points (p=0.74) and a wrong‑signed correctness‑versus‑conflict association (r=+0.15). A proof‑preserving relabeling lowers accuracy in all five clusters for one model (mean ‑93 points) but not another, exposing model‑surface sensitivity. In a preregistered extension, provider‑reported completion‑token spend does not consistently increase with the proxy after accounting for formula length and censoring. At 16k, the reasoning model spends more on proof‑easy matched formulas and exhausts its budget on the solver‑easiest UNSAT family; the 32k C1 gap is absent. These scoped dissociations concern verdict accuracy and observed token spend, not certificate solving, exact proof length, or allocation efficiency.
Authors:Amez Amanj Ali, Kuo-Kun Tseng
Abstract:
This paper addresses key technical challenges in current large language model (LLM) agent applications, including long‑horizon planning, sparse reward attribution, and dynamic environmental interaction, by designing and optimizing an intelligent agent workflow. The proposed architecture is based on the synthesis of core AI paradigms: Visual, Language, Generative, Graph, Multimodal, Reinforcement, and Agent Intelligence. Unlike conventional baseline models that rely on static prompting and lack robust perception‑action loops, our approach introduces a Partially Observable Markov Decision Process (POMDP) routing mechanism. This mechanism is augmented with an internal, self‑correcting reward model that evaluates decision trajectories before execution. By integrating multimodal inputs and advanced reinforcement learning principles (such as proximal policy optimization and value function approximation), the agent maintains long‑term structural memory and dynamically adapts its reasoning pathways to mitigate error accumulation. Empirical experiments on the ALFWorld embodied simulation environment and the WebShop online navigation benchmark demonstrate a 24.5% absolute improvement in task success rate and trajectory efficiency over mainstream baselines like the standard ReAct framework. Comprehensive ablation studies confirm the significant contribution of the reward‑driven critique module in suppressing hallucination rates. This research bridges theoretical foundations of reinforcement learning and graph‑based memory with autonomous agent workflows. Ultimately, the resulting architecture offers a practical, scalable reference framework for developing artificial intelligence technologies in complex, multi‑step autonomous systems. Code is available at https://github.com/01Amez/RLAW_Implementation.
Authors:Keren Zhu
Abstract:
Large language models may make precise but dormant algorithmic problems practical to revisit, and may expose new paths toward fundamental ones. We demonstrate this possibility through Prim‑Dijkstra routing, a classic VLSI problem whose terminal‑only Manhattan complexity remained open despite decades of practical work. We prove weak NP‑completeness, derive a continuous cost‑radius tradeoff with a balanced (2,2) guarantee, and build HP‑RCRST, a height‑partition‑based multi‑mode solver. On 28 development instances, its stronger modes Pareto‑dominate the published‑method union on 23 and tie on five. The case shows how conflicting conjectures, counterexamples, formal checks, and implementation can reopen neglected questions. Code and reproducibility materials are available at https://github.com/CODA‑Team/hp‑rcrst.
Authors:Neel Somani
Abstract:
Mathematicians distinguish proofs that explain, simplify, or introduce a nonstandard route, but these judgments are difficult to operationalize. We study a deliberately narrower construct: time‑relative proof‑route nonstandardness in formal mathematics. For a Lean theorem, PriorProof extracts the dependency footprint of its elaborated proof term and scores the weighted surprisal of that footprint under a retrieval‑conditioned, hierarchically smoothed prior built only from an earlier quarterly snapshot of Mathlib. The method requires no hand‑built technique ontology and no human labels: statement retrieval is learned from proof‑derived contrastive pairs, while the scored object is read mechanically from proof terms. In a blinded topology study, 100 presentations collapse to 76 distinct underlying pairs: 12 canonical contrasts shown three times for consistency screening and 64 distinct stratified pairs. Against the majority of three retained domain raters, PriorProof agrees on 53/76 pairs (69.7%, Wilson 95% CI 58.7‑78.9%), including 11/12 canonical pairs (91.7%, 64.6‑98.5%) and 42/64 stratified pairs (65.6%, 53.4‑76.1%). Score‑gap quartiles are nonmonotone after repeat collapse; the endpoints are 12/19 (63.2%, 41.0‑80.9%) in the smallest‑gap bin and 16/19 (84.2%, 62.4‑94.5%) in the largest, supporting an endpoint‑calibration tendency rather than a resolved staircase. The best language‑model condition agrees on 60/76 pairs (78.9%, 68.5‑86.6%); on paired outcomes, PriorProof alone is correct on 8 pairs and the model alone on 15 (exact two‑sided McNemar p = 0.210), so the difference is not established at this sample size. We therefore present PriorProof not as a replacement for expert or model judgment, but as a decomposable, time‑anchored signal whose score gap provides an interpretable reliability indicator.
Authors:Mohammad Arvan, Amber E. Osterholt, Bailee Rue, Yuvaneswaren R. Sureshbabu, Krishna R. Patel, Rebecca T. Feinstein, Bethany C. Bray, Niranjan S. Karnik
Abstract:
Introduction. Clinical and Translational Science Award (CTSA) programs must document their scholars' research impact, but assembling each scholar's record by hand takes staff an estimated 15 hours and does not scale to a full cohort. An artificial intelligence (AI) agent could serve as a tool to gather scholar data across platforms and disciplines. Methods. We built a human‑in‑the‑loop AI agent that assembles a dossier of sourced evidence for each scholar and drafts one‑sentence Translational Science Benefits Model (TSBM) impact summaries for staff review. We evaluated it in the impact‑reporting workflow of one CTSA hub across 10 career‑development (KL2/K12) scholars. Two evaluation staff independently coded all 507 findings as accept, edit, or reject; the primary measure was the unanimous usable rate, defined as the share both accepted or edited. Results. Both reviewers accepted or edited 81.7% of the agent's findings. Reviewers each spent a median of 14 minutes per scholar, replacing an estimated 15 hours of manual assembly. Inter‑rater agreement was moderate (Cohen's kappa 0.43 on the usable‑versus‑reject decision). A profile discovery study found the agent's recall close to human search. The agent's impact evidence spanned all four TSBM domains, and about a third of the reviewed findings fell in non‑scholarly categories that routine processes tend to miss. Reviewers rated synthesis accuracy 4.5 and usefulness 4.8 on a 5‑point scale. Conclusions. A human‑in‑the‑loop AI agent can serve as the first‑pass author of a scholar's impact record, shifting staff from collecting and writing to reviewing, and making cohort‑scale impact reporting feasible.
Authors:Maxence Noble, Marie Scheid, Yazid Janati, Eric Moulines, Alain Durmus
Abstract:
Over the past few years, diffusion‑based Schrödinger bridge models have been proposed to approximate optimal transport dynamics between two prescribed boundary distributions, with successful applications to generative modeling. More precisely, these methods aim to estimate a path measure whose initial and terminal marginals match the two boundary distributions, while minimizing the Kullback‑Leibler divergence with respect to a reference Markov process. In this work, we consider the generalized Schrödinger bridge problem, in which the reference process is a twisted Brownian motion, that is, a Feynman‑Kac transform of a Brownian motion induced by a time‑dependent differentiable potential. Building on the Iterative Markovian Fitting (IMF) paradigm, and in particular on its special case Diffusion Schrödinger Bridge Matching (DSBM), which corresponds to the zero potential case, we introduce Twisted Schrödinger Bridge Matching (TSBM), a diffusion‑based method designed to handle both continuous‑ and discrete‑time potentials. Unlike previous approaches, TSBM provides a rigorous extension of the IMF scheme to the generalized Schrödinger bridge problem. This derivation leads to a new bridge‑matching loss that depends explicitly on the gradient of the potential and recovers the DSBM objective when the potential vanishes, yielding improved performance. We further introduce trajectory‑based variance‑reduction techniques that substantially stabilize optimization and may be useful beyond the present setting. Finally, we empirically demonstrate the benefits of TSBM for trajectory inference across increasingly high‑dimensional settings, including crowd navigation and single‑cell data. Code available at https://github.com/maxencenoble/twisted‑sb‑matching.
Authors:Patrick Cooper, Alvaro Velasquez
Abstract:
An agent acting under partial observability must decide when to gather information and which observations are worth their cost. Standard POMDPs value information only through its eventual effect on reward. The ρ‑POMDP framework instead rewards uncertainty reduction directly, through a belief‑dependent utility ρ, but in practice both the choice of ρ and the weight placed on it are tuned by hand for every task. We show that active inference removes this tuning entirely. Minimizing Expected Free Energy (EFE) is exactly equivalent to solving a ρ‑POMDP whose utility is expected information gain, and the exploration weight is fixed at w=1 because the variational bound expresses pragmatic and epistemic value in the same units (nats). We prove this equivalence for observe‑then‑commit POMDPs and extend it to factored observation POMDPs, a broader class that covers interleaved observe‑act problems such as non‑destructive testing and mobile sensing, where gathering information leaves the hidden state unchanged. Experiments support the theory. Across environments ranging from the classic Tiger problem to RockSample and a new Structural Inspection benchmark with over 65,000 states, the untuned weight matches or outperforms reward‑only planning at the same horizon, avoids the over‑exploration of bonuses tuned per task, and sits near the reward‑maximizing knee of the success‑reward Pareto frontier. The practical payoff is an exploration objective that works out of the box. In applications such as fault detection and medical screening, where every test has a price and every missed fault has a cost, EFE supplies a belief‑dependent utility that is derived rather than tuned.
Authors:Varun Yerram, He He, Eunsol Choi
Abstract:
Continuous Chain‑of‑Thought methods replace verbose reasoning traces with a short sequence of dense latent representations. Earlier continuous CoT methods indirectly supervise the latent representations such that its final state match that of verbose reasoning traces, requiring autoregressive, slow generation during training. We introduce C‑MTP, a simpler, faster direct supervision approach that models each latent as an average of the embeddings in the CoT traces to be compressed. Our approach outperforms a prior direct supervision method that approximates the distribution of compressed tokens, and performs competitively to slower indirect supervision approaches in existing evaluation setup with simplified CoT traces (less than 100 tokens). Lastly, we extend the evaluation of Continuous CoT methods to complex tasks with longer reasoning traces (\ge few hundreds reasoning tokens). We find both direct and indirect supervision training methods perform poorly (roughly 65% performance drop) in this setting, revealing the limitations of current continuous CoT methods. The code and checkpoints are released at https://github.com/Varun221/cmtp_research
Authors:Nicole Feng, Ioannis Gkioulekas, Keenan Crane
Abstract:
We describe a method for computing signed distance to point clouds that allows fast pointwise evaluation at arbitrary spatial resolution. As input, our method takes a point cloud with normals; as output, it provides an analytical parameterization that allows queries of signed distance to the approximate underlying surface at arbitrary points ‑ simultaneously providing reconstruction and distance. Our key idea is to reconstruct shapes by locally fitting point clouds with tori, which have closed‑form signed distance functions. Tori are fitted in a feed‑forward manner, using a pre‑trained network to output per‑point curvature and shift parameters. Importantly, our method does not require costly global optimization or spatial discretization, and is easily parallelizable. Underlying our method is a new theory that unifies signed distance with the classic reconstruction methods of winding numbers and Poisson surface reconstruction. We use our method to compute signed distance to point clouds arising from photogrammetry, meshes, 3D Gaussians, and neural implicits. Our method allows point clouds to be used directly in applications, without explicit surface reconstruction: as examples, we take offsets of point clouds, apply morphological and Boolean operations, and directly visualize offset surfaces using sphere tracing.
Authors:Yunwei Li, Shengjie Fu, Chunrong Chen, Chengxiang Zhao, Yuchen Fan, Mingyu Zhu, Yanchao Xu, Yuxin Zhang, Lan Yang, Chuzhao Li, Jie Ji, Yi He, Abhijit Sarkar, Akash Sonth, Hong Wang, Jun Li
Abstract:
Safety validation at signalized intersections remains a critical bottleneck for the deployment of autonomous driving systems (ADS), as these scenarios involve dense heterogeneous traffic, contested right of way, and long‑tail safety‑critical interactions, posing significant challenges to the Safety of the Intended Functionality (SOTIF). Existing naturalistic driving datasets often suffer from geographical homogeneity, sparsity of safety‑critical events, and lack of semantic risk annotations, which limit the evaluation of algorithmic generalizability and targeted SOTIF verification. To address these gaps, this paper introduces SinD 2.0, a large‑scale drone‑based intersection dataset dedicated to cross‑domain ADS safety analysis. The main contributions of SinD 2.0 are: (1) Cross‑domain diversity: It covers six signalized intersections across four Chinese cities, capturing distinct intersection topologies and regional driving behavior characteristics; (2) High‑density risk interactions: A total of 32,682 safety‑critical events are extracted via surrogate safety measures, significantly enriching the density of boundary test scenarios; (3) Hierarchical semantic annotations: Besides integration with high‑definition (HD) maps and Signal Phase and Timing (SPaT) data, it provides multi‑dimensional semantic labels including traffic violations, high‑risk interactions, visual shielding, and narrow feasible areas; (4) Full‑stack testing toolchain: It supports automated scenario extraction, prediction‑only evaluation, open‑loop replay, reactive closed‑loop testing, and photorealistic rendering. Benchmark experiments demonstrate that SinD 2.0 exhibits significant domain shifts across cities, and the semantic risk subsets can effectively expose the performance limitations of ADS algorithms. The dataset, annotations, and testing toolchain are available at https://github.com/SOTIF‑AVLab/SinD/tree/main.
Authors:Hyeonjoong Jang, Dongyoung Choi, Donggun Kim, Woohyun Kang, Min H. Kim
Abstract:
We propose a splat‑based 3D scene reconstruction method from RGB‑D input that effectively handles extreme motion blur, a frequent challenge in low‑light environments. Under dim illumination, RGB frames often suffer from severe motion blur due to extended exposure times, causing traditional camera pose estimation methods, such as COLMAP, to fail. This results in inaccurate camera pose and blurry color input, compromising the quality of 3D reconstructions. Although recent 3D reconstruction techniques like Neural Radiance Fields and Gaussian Splatting have demonstrated impressive results, they rely on accurate camera trajectory estimation, which becomes challenging under fast motion or poor lighting conditions. Furthermore, rapid camera movement and the limited field of view of depth sensors reduce point cloud overlap, limiting the effectiveness of pose estimation with the ICP algorithm. To address these issues, we introduce a method that combines camera pose estimation and image deblurring using a Gaussian Splatting framework, leveraging both 3D Gaussian splats and depth inputs for enhanced scene representation. Our method first aligns consecutive RGB‑D frames through optical flow and ICP, then refines camera poses and 3D geometry by adjusting Gaussian positions for optimal depth alignment. To handle motion blur, we model camera movement during exposure and deblur images by comparing the input with a series of sharp, rendered frames. Experiments on a new RGB‑D dataset with extreme motion blur show that our method outperforms existing approaches, enabling high‑quality reconstructions even in challenging conditions. This approach has broad implications for 3D mapping applications in robotics, autonomous navigation, and augmented reality. Both code and dataset are publicly available on https://github.com/KAIST‑VCLAB/gs‑extreme‑motion‑blur.
Authors:Haoru Tan, Wang Wang, Sitong Wu, Xiuzhe Wu, Yangtian Sun, Chirui Chang, Shaofeng Zhang, Xiaojuan Qi
Abstract:
We revisit dataset distillation from an outcome‑centric perspective. Rather than aligning process surrogates (per‑step gradients or training trajectories), Influence Matching (Inf‑Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample‑level influence estimator that quantifies parameter shifts from adding or removing data, without time‑consuming inverse‑Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first‑order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf‑Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny‑ImageNet (IPC=10), Inf‑Match attains 31.5%, a +4.7% improvement over NCFM. Beyond classification, Inf‑Match scales to vision‑language distillation on Flickr30K, outperforming strong process‑matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5%. The code will be released via https://github.com/hrtan/infmatch.
Authors:Pengxu Chen, Yao Zhu, Guangming Zhu, Jun Sheng, Jincai Huang, Xiangyang Ji, Liang Zhang
Abstract:
Large vision‑language models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding. However, they remain prone to hallucinations, generating responses that are inconsistent with the visual evidence. Existing mitigation methods largely address language‑prior bias or cross‑modal imbalance, while progressive visual degradation across perception and memory remains underexplored. In this work, we propose Saliency‑Driven Perceptual Realignment (SDPR), a training‑free framework that mitigates the degradation of visual awareness throughout inference. Specifically, we first introduce saliency‑driven attention redistribution to release attention hijacked by non‑semantic sink tokens, thereby recovering critical visual evidence. Second, we identify spatial distortion in the KV cache and propose saliency‑driven cache alignment to preserve query‑relevant visual features during generation. Finally, we introduce prior‑constrained contrastive decoding to penalize unfaithful predictions induced by dominant language priors. Our proposed SDPR is robust against hallucinations due to its holistic alignment of visual awareness across the entire generative trajectory. Extensive experiments across diverse LVLM architectures show that SDPR outperforms state‑of‑the‑art methods on both hallucination and general‑purpose benchmarks, requiring no additional training and incurring minimal runtime overhead. The code is available \hrefhttps://github.com/PengSyuChen/SDPR\colorbluehere.
Authors:Yao Huang, Yitong Sun, Huanran Chen, Ruochen Zhang, Shouwei Ruan, Ranjie Duan, Maoxun Yuan, Yinpeng Dong, Hui Xue, Xiaochun Cao, Xingxing Wei
Abstract:
Despite the impressive generative capabilities of text‑to‑image diffusion models, they remain vulnerable to implicit sexual prompts, where subtle cues disguised as benign terms or adversarial tokens unexpectedly generate the inappropriate content due to model biases or latent correlations in training data. Existing safety mechanisms face fundamental limitations: detection methods primarily identify explicit content and fail to capture implicit malicious intent, while mitigation approaches rely on static negative prompts inadequate for diverse implicit scenarios. To address these challenges, we propose UniNDM, a unified noise‑driven framework that rethinks safety mechanisms through the lens of noise dynamics in diffusion processes. Our key insight is that early‑stage predicted noise exhibits inherent separability between normal and sexually explicit content, which we theoretically demonstrates quadratically increasing semantic concentration with timestep. Leveraging this property, we develop a lightweight noise‑based detector achieving superior accuracy with virtually no computational overhead. For mitigation, we introduce noise‑enhanced adaptive negative guidance: dynamically generating context‑specific negative prompts via large language models to handle diverse implicit content, while optimizing initial noise by suppressing attention concentration on explicit tokens to provide comprehensive protection. Besides the U‑Net‑based diffusion models, we further extend our framework to emerging Diffusion Transformer architectures through region‑constrained semantic guidance tailored for their unified multimodal attention. Comprehensive experiments across U‑Net models and DiT models on both natural and adversarial datasets demonstrate substantial improvements over state‑of‑the‑art methods, including SLD, UCE, Safree, etc. Our code is publicly available at https://github.com/Aries‑iai/UniNDM.
Authors:Robert Wijaya, Md. Tanvir Hossain, Amanda Kau, Ngai-Man Cheung
Abstract:
Generalised object counting aims to estimate the number of instances of an arbitrary object category from a single image, but many recent methods can struggle on structurally complex objects due to limited spatial modelling. We present UpCount, a class‑agnostic counter designed to better preserve spatial structure. UpCount strengthens the visual representation by extracting multi‑layer features from a ViT‑B/16 encoder and reassembling them into a refined multi‑scale pyramid that is spatially refined using Dense Prediction Transformers and FeatUp, yielding features with improved structural and spatial sensitivity; a proposal‑‑verification counting head then identifies repeated patterns and produces a density map for the final count. On FSC‑147, UpCount achieves 12.39 MAE and 100.89 RMSE on the test set, and it transfers effectively to vehicle counting on CARPK (6.27 MAE, 8.79 RMSE). Code: https://github.com/r28112072‑rgb/upcount
Authors:Roberto Pietrantuono, Antonio Guerriero, Pouya Sattari
Abstract:
Event Argument Extraction (EAE) converts documents into structured event records by identifying argument spans and assigning them schema‑defined roles. Document‑level EAE is challenging due to long‑range dependencies between triggers and arguments, cross‑sentence context, and strict role constraints, which often lead to boundary errors, uncertainty in roles, and inconsistencies with restricted schemas. In this paper, we study whether mid‑sized open LLMs can perform schema‑constrained EAE reliably at the document level on MAVEN‑ARG. Our approach combines (i) role‑set injection in prompts for schema compliance, (ii) parameter‑efficient supervised fine‑tuning (LoRA) using the same JSON‑only interface used at inference, and (iii) deterministic decoding with post‑processing that validates JSON, filters invalid roles, de‑duplicates arguments, and aligns spans to the document window. Under the official MAVEN‑ARG evaluator, fine‑tuned mid‑sized open models outperform previously reported GPT baselines across mention, entity‑coreference, and event‑coreference evaluations; our best model (Phi‑4, 14B) reaches 42.39% F1 at the event‑coreference level. Code to reproduce experiments is publicly available at https://github.com/dessertlab/EAE/.
Authors:Tianshuai Hu, Yangyi Zhong, Zeying Gong, Lingdong Kong, Xiaodong Mei, Guoyang Zhao, Xiaolu Liu, Song Wang, Rong Li, Junwei Liang
Abstract:
Vision‑Language Navigation in dynamic, human‑centric environments exposes a fundamental tension: linguistic reasoning is slow and deliberative, whereas safe, socially compliant planning should be instant and reactive. The resulting observation staleness is safety‑critical: a maneuver chosen during inference can already be unsafe by the time it executes. We observe that, long before a VLM finishes its inference, its intermediate hidden states already encode action‑relevant intent. We propose SPARK‑VLN, a dual‑system framework for dynamic social VLN that streams the slow VLM reasoner's knowledge to a fast flow‑matching expert planner throughout token generation, providing fresh and evolving guidance during inference. This design is realized by three modules: a Token‑Wise Hidden Streamer that extracts intermediate hidden states along the token generation process, a Sequence‑to‑Slot Latent Bridge that projects them into fixed‑size latent slots, and an Evolving Latent Conditioner that infuses them into the expert planner. We also introduce a human‑centric benchmark suite for dynamic social vision‑language navigation that keeps pedestrians and the robot active throughout inference and reports navigation success, social compliance, human collisions, and explicit staleness statistics. Across these settings, SPARK‑VLN mproves navigation success and social compliance while sustaining inference efficiency. Webpage: https://hutslib.github.io/SPARK‑VLN/.
Authors:Yuqi Zhang, Yadan Luo, Xiangyu Sun, Fengyi Zhang, Zi Huang, Xin Tan
Abstract:
High‑quality 3D scene assets are critical for embodied applications such as robotic manipulation, navigation, and simulation. Despite their strong object priors, recent single‑image 3D generation models such as SAM3D remain insufficient for real‑world scenes, where severe occlusions, redundant observations, and cross‑view inconsistencies make reliable scene generation challenging. We introduce Scene‑SAM3D, a training‑free framework that extends SAM3D from single‑view object generation to calibrated multi‑view scene asset generation. Scene‑SAM3D selects a compact set of complementary views, reducing observation redundancy while providing additional evidence for regions occluded in individual views. Based on the selected views, it performs step‑efficient latent velocity fusion to integrate multi‑view evidence and suppress cross‑view conflicts in canonical space. Finally, a lightweight rigid‑object Gaussian optimization refines the scene layout within 200 iterations while preserving the generated object geometry. Experiments on Replica and ScanNet++ demonstrate consistent improvements at both instance and scene levels, with our method reducing scene‑level CD by 43.8% on Replica and 30.9% on ScanNet++, while cutting flow‑model sampling FLOPs and wall‑time latency by nearly 20% under the same multi‑view setting. Code will be released at https://github.com/xibi777/Scene‑SAM3D.
Authors:Ayşe Özlem Çalışkan, Jordi Sanchez-Riera
Abstract:
People with low vision often face challenges in performing everyday tasks that require interpreting visual information. We present VisionAssist, an open‑source mobile application designed to improve independence by providing AI‑powered visual assistance through a smartphone. The application integrates three complementary functionalities within a single interface. First, it enables users to locate specific objects by analyzing the live camera feed. Second, it generates spoken descriptions of captured images, allowing users to identify visual content such as food labels, documents, and everyday objects. Third, it integrates with the smartphone's contacts and calendar to facilitate emergency calls and provide voice‑based reminders. The application supports hands‑free interaction through voice commands and delivers all feedback using text‑to‑speech synthesis, making it fully accessible to users with visual impairments. By combining multiple assistive services into a unified platform and releasing the project as open‑source software, the proposed solution aims to encourage community contributions and accelerate the development of accessible technologies. The source code is publicly available at: https://github.com/AOzlemC/LowVisionProject.git
Authors:Francesco Karim Vicidomini
Abstract:
Bürger et al. (2024) demonstrated that truth representations in large language models are universal across statement polarity but reside within a multidimensional subspace. We extend this framework along three questions: how the dimensionality of the subspace depends on the model's knowledge, which architectural component builds the truth direction, and what the direction is a mixture of. In Part I, a training‑free directional probe derived from the SVD of hidden‑state minimal pairs shows that the dimensionality of truth is knowledge‑dependent: the signal concentrates on a single axis for known facts and diffuses as knowledge decreases. In Part II, a relational law emerges across multiple model families: attention propagates truth frames, the feed‑forward network opposes the current block's frame, and post‑peak decay is causally attributed to the SwiGLU value stream. Furthermore, per‑category truth axes form a semantically signed arrangement that converges across families. Stress tests expose a sign instability in this orientation, which we repair with a spectral consensus gauge to sharpen the convergence into a knowledge‑gated law. Finally, a replication campaign on Gemma‑2‑2b, extending our decomposition tools to accommodate its sandwich normalization, confirms these laws and attributions. We quantify the knowledge gate as classical attenuation and isolate a stable, model‑specific private geometry.
Authors:Florian Schmid, Paul Primus, Alexander Fichtinger, Tara Jadidi, Tobias Morocutti, Gerhard Widmer
Abstract:
This paper presents RealDESED, a real‑world domestic sound event detection (SED) benchmark comprising 5,710 audio recordings collected by 652 participants in their homes. Each recording is between 15 and 35 seconds long and contains temporally precise annotations for 15 common domestic sound classes. In contrast to existing SED datasets, which typically rely on simulated soundscapes or broad web‑crawled audio, RealDESED consists exclusively of recordings captured in natural domestic environments, reflecting realistic variability in recording devices, device placement, acoustic conditions, background sounds, and naturally occurring event co‑occurrences. A distinguishing characteristic of the dataset is its multi‑annotator labeling scheme, where each recording is independently annotated by multiple annotators, while the validation and test sets undergo an additional review process to ensure high annotation quality and reliable benchmarking. Furthermore, the dataset provides rich metadata, including recording device, device placement, environment labels, and textual scene descriptions. We establish a strong transformer‑based baseline and investigate annotation aggregation strategies, post‑processing methods, long‑form inference, and the impact of recording metadata on model performance. Our baseline achieves a macro‑averaged PSDS1 score of 0.731 on the test set. We believe RealDESED provides a valuable benchmark for developing and evaluating robust SED systems under realistic domestic conditions, helping to bridge the gap between current research benchmarks and real‑world deployment.
Authors:Zhenhua Zhao, Jihao Long
Abstract:
We introduce the Deep Second‑Order Stochastic Residual Method (D2SRM) for high‑dimensional, Hessian‑dependent fully nonlinear parabolic PDEs. A single scalar space‑‑time network generates derivative‑consistent approximations of the solution, gradient, and Hessian, which are trained jointly through second‑order Brownian one‑step residuals and terminal value and gradient penalties. For globally Lipschitz equations with identity diffusion and sufficiently weak Hessian coupling, we establish well‑posedness in a Brownian occupation space and develop a population‑level convergence theory. Under additional regularity, an a posteriori estimate bounds the squared full‑jet occupation error of any admissible candidate by the time step and its population objective. For approximate population minimizers, the error bound separates time discretization, neural approximation, and population suboptimality; when the latter two terms are O(h), the full‑jet occupation norm is O(h^1/2). Experiments on a 100‑dimensional manufactured benchmark compare terminal treatments, probe Hessian couplings inside and outside the proved small‑gain range, and show decreasing errors as the time step decreases. The code is available at https://github.com/ZZHPKU/D2SRM.
Authors:Victor Gong, David Guecha
Abstract:
We describe DS@GT's submission to the eRisk 2026 Task 1 challenge on conversational depression screening, in which systems interview LLM personas that simulate individuals with varying depression profiles and produce a Beck Depression Inventory II (BDI‑II) score plus four key symptoms per persona, without directly asking sensitive mental health questions. Our pipeline evolved through three stages: a monolithic single‑model prototype to start off, a baseline multi‑agent architecture that separates conversational interviewing from BDI‑II scoring under a coordinating orchestration layer, and a final hybrid configuration that replaces the paid GPT‑5‑nano interviewer with the open‑source Gemma 27B. To offset the model's weaker reasoning and instruction‑following, the hybrid adds three algorithmic components: a precomputed dialogue tree that standardizes interview openers and follow‑ups, a reliability‑weighted consensus aggregation inspired by the Weaver framework, and a cluster‑based imputation step for unprobed symptoms. We submitted three fully automated runs across all 20 personas, with Run 1 from the paid baseline and Runs 2 and 3 from the hybrid. Hybrid Run 3 achieved an ADODL of 0.9063, ranking 3rd among all complete‑submission runs and placing DS@GT 2nd among the 21 teams overall, while outperforming our paid baseline Run 1 (0.8841) at roughly one‑quarter of the per‑persona API cost. These results support our central hypothesis that with sufficient algorithmic supervision, a weaker open‑source model can compete with a stronger proprietary model in the conversational interviewer role. Our source code is available at https://github.com/dsgt‑arc/erisk‑task1‑2026.
Authors:Tavish Mankash, Vardhaman Kalloli, Keshava Prasad, Deepan Muthirayan
Abstract:
OpenLanguageModel (OLM) is an open‑source PyTorch library for building and pretraining small language models while keeping their machinery visible. In OLM, model code reads like the architecture: components are ordinary modules, while Block, Residual, Repeat, and Parallel describe how they are wired. The resulting model can move unchanged from a teaching notebook to a complete pretraining run or a research ablation. OLM connects this readable model layer to tokenizers, local and streaming datasets, optimization, mixed precision, callbacks, checkpoints, and hardware‑aware CPU, single‑GPU, and single‑node multi‑GPU execution. We demonstrate the full path by tracing GPT‑2 from diagram to code, launching a FineWeb‑Edu training script, replacing one attention component, and letting AutoTrainer configure the available machine. The package includes 27 presets across nine familiar model families and documentation that progresses from LM fundamentals to architecture research. Validation shows close agreement with independent reference implementations, 90.6% four‑GPU weak‑scaling efficiency for a 348M‑parameter workload, compact architecture edits, and positive early usability results. OLM is MIT‑licensed and available through PyPI, GitHub, and its documentation site.
Authors:Haitong Tang, Haipeng Liu, Yang Wang
Abstract:
Object removal aims to eliminate target objects specified by a mask while preserving visual consistency with the surrounding regions. Existing methods typically rely on contextual information from surrounding regions. However, in dense scenes where the surrounding regions contain instances visually similar to the removal target, such reliance often leads to semantic interference, resulting in incomplete removal. This problem arises from erroneous information propagation in the attention space, where masked queries tend to align with such instances due to global similarity matching in self‑attention. To address this challenge, we propose a Diffusion‑based Object Removal framework for dense Scenes, dubbed DORS, built upon a Dynamic Attention Routing mechanism comprising two complementary components: Instance‑Filtered Attention (IFA), which suppresses misleading semantic information from similar instances through dynamically constructed mask‑guided attention constraints, and Context‑Guided Routing (CGR), which dynamically routes complementary scene information to maintain visual consistency. We further introduce DOR‑Bench, a benchmark tailored for object removal in dense scenes. Extensive experiments demonstrate that DORS outperforms state‑of‑the‑art methods, particularly in reducing incomplete removal and duplicate artifacts. The code will be available at https://github.com/httang1224/DORS.
Authors:Yoonseok Choi, Eun-Gyu Ha, Daniel Kim, Mohammed A. Al-masni, Ming-Hsuan Yang, Dong-Hyun Kim
Abstract:
Magnetic Resonance Imaging (MRI) is often acquired with anisotropic resolution to reduce scan time, producing stair‑step artifacts along the through‑plane direction. In through‑plane MRI super‑resolution, an efficiency‑fidelity trade‑off arises: feed‑forward regressors are fast but oversmooth at large slice‑thicknesses, while sampling‑based methods improve fidelity at high inference cost. We propose DRIFT, a two‑stage thickness‑conditioned rectified flow framework for through‑plane MRI super‑resolution with continuous input slice‑thickness. Stage 1 employs an Anatomical Projection Network (APN) to map low‑resolution patches to a coarse high‑resolution manifold, providing a deterministic anatomical initialization that shortens the residual transport of Stage 2 and stabilizes slice‑wise refinement. Stage 2 refines details via rectified flow and introduces a Physics‑Aware Difficulty (PAD) metric derived from slice‑thickness induced through‑plane bandwidth deficit to guide an Adaptive Integration Scheduler (AIS), allocating ODE steps by thickness. A Consistent Endpoint Trajectory Alignment (CETA) loss enforces thickness‑consistent reconstructions. Experiments show that DRIFT outperforms super‑resolution baselines while reducing inference cost. Code, models, and interactive demos are available at https://yoonseokchoi‑ai.github.io/drift‑eccv2026/.
Authors:Chen Chen, Zhehuai Chen
Abstract:
Long‑horizon AI agents are becoming increasingly capable, yet their interaction with users remains surprisingly thin. In most workflows, users give an initial instruction, receive only selective textual updates, and lose a clear sense of what the agent is doing or when to step in. This leaves a missing part in the current agent ecosystem: an always‑on Jarvis‑style mediator that keeps the agent continuously reachable to the user. Such a mediator should support real‑time spoken interaction with the user, answer questions without interrupting the worker, proactively report progress or confusion, and inject user guidance back into the agent's execution when useful. In this work, we introduce JarvisBench, a benchmark for measuring the dual value of mediation in long‑horizon agent workflows. JarvisBench contains two complementary tracks: an agent‑collaboration track that measures whether mediation improves downstream task completion, and a user‑interaction track that measures whether mediation makes ongoing execution more understandable, responsive, and accessible to users. We instantiate the benchmark with a modular reference Jarvis prototype and evaluate it on 34 text‑only WildClaw tasks executed in OpenClaw. Preliminary results with GPT‑5.5, Claude Opus 4.7, Gemini‑based, and GPT‑based worker agents suggest that Jarvis‑style mediation can provide trace‑grounded responses to user questions and improve task performance when sparse user guidance is injected at appropriate moments. The results also show that effectiveness depends strongly on the mediator's LLM brain, highlighting both the promise of this missing middle layer and the need for broader community effort. Demo page https://cchen1436.github.io/jarvis
Authors:Han Wang, Zijun Wang, Shuoshuo Xue, Rui Cao, Fengjiao Cheng, Xiaodan Liang, Roy Ka-Wei Lee
Abstract:
Action‑conditioned world models are a key component of embodied AI, serving as scalable policy evaluators that reduce reliance on expensive real‑world rollouts. To accurately capture diverse action‑induced dynamics, such models should satisfy three key objectives‑Physical Plausibility (P), Action Adherence (A), and Visual Fidelity (V), collectively referred to as PAV‑while remaining robust to both in‑distribution (ID) expert demonstrations and out‑of‑distribution (OOD) actions. However, existing methods primarily rely on ID action‑video pairs and pixel‑level reconstruction losses, which do not explicitly optimize PAV objectives and generalize poorly beyond expert data. To address this, we propose PAVXploreRL, a reinforcement learning framework built on a pretrained latent world model that explicitly optimizes PAV objectives through reward‑driven training. To improve action generalization, our method jointly leverages ID trajectories and noise‑driven OOD action exploration, without paired video supervision. Experiments show that PAVXploreRL consistently outperforms pretrained baselines, achieving a 5.6% average gain across benchmarks and producing higher‑quality PAV properties. As a policy evaluator, it also yields more reliable performance estimates and reduces the overestimation bias of prior expert‑only world models such as Ctrl‑World. Code: https://github.com/Social‑AI‑Studio/PAVXploreRL
Authors:Georgii Kliukovkin
Abstract:
The startup latency of a model‑serving pod on Kubernetes is dominated by one step: delivering the model weights. As models reach the hundred‑gigabyte weights of large language models, cold‑start delivery time governs the economics of autoscaling and scale‑to‑zero, yet the dominant mechanisms remain ad‑hoc downloads from object storage, with none of the pull caching, digest addressing, or verification Kubernetes provides for container images. We analyze the delivery paths available to a Kubernetes serving platform along two axes: which component pulls the artifact, and whether any admission‑time verifier can bind the deployed reference to the arriving bytes. We validate the analysis upstream in KServe, a widely deployed CNCF model‑serving platform, by implementing two new delivery paths: oci+native://, which mounts model images as Kubernetes image volumes (KEP‑4639), merged upstream, and oci+fetch://, which pulls OCI artifacts inside the storage initializer, under review. We report, to our knowledge, the first controlled comparison of model delivery paths in a Kubernetes serving platform (modelcar sidecars, native image volumes, object‑storage download) on artifacts sized to fp16 weights of 1B‑, 7B‑, and 70B‑class models (2‑140 GB). Node‑cached OCI delivery makes warm replica addition size‑independent: 11.7 s for a 70B‑class artifact versus 40.7 minutes of re‑download over object storage, a 208x difference, while the first cold pull costs up to 2x a plain download, localized to containerd's blob‑write‑then‑unpack double pass. For models on s3://, gs://, or hf:// URIs, where no admission‑time verifier observes the bytes, we present a serving‑time integrity design proposed to the KServe community: digest pinning and OpenSSF model‑signing enforcement in the storage initializer. Streaming hash verification during download adds under 0.1% to delivery time; a post‑download pass adds up to 53%.
Authors:William Huang, Ruofei Du, Yang Zhang
Abstract:
Travel planning requires balancing interacting goals and constraints across time and space. Current AI travel tools provide limited support for encoding these constraints and understanding how generated travel plans may fail users. We present AlterAtlas, an interactive travel planning system that supports high‑fidelity itinerary validation and revision through persona‑based simulations grounded in geospatial information. AlterAtlas models travelers as editable personas, generates candidate itineraries from prioritized places of interest, and simulates how different personas would experience each plan. Simulations expose route‑level tradeoffs, temporal user states (e.g., fatigue, hunger), and mismatches between plans and user preferences to allow users to iteratively refine both itineraries and user personas. An expert evaluation of 51 paired itineraries demonstrates that simulation‑guided revisions significantly improve plan‑persona alignment. Furthermore, a within‑subjects study (N=11) reveals that AlterAtlas empowers users to uncover hidden constraints, fluidly compare alternatives, and build trust in their final plans. Our results suggest that simulation‑based validation is a powerful, transparent interaction layer for AI‑assisted travel planning.
Authors:Yuhan Liu, Xinyu Zhang, Litao Liu, Abdeslam Boularias
Abstract:
Vision‑Language‑Action (VLA) policies offer strong general‑purpose manipulation priors, but often fail on tight‑tolerance, contact‑rich assembly due to long‑horizon credit assignment and subtask coupling: a state that is geometrically successful for the current skill can be brittle for downstream skills. We show this failure mode in residual reinforcement learning (RL) over a frozen VLA base policy: constant sparse success rewards improve each subtask in isolation yet yield little or no gain when skills are chained, because terminal state quality is uncontrolled. We propose Foresight Residual RL, which optimizes handoff quality by augmenting each subtask's sparse success reward with an offline‑estimated foresight value ‑‑ the probability of future subtask success conditioned on the terminal state of the current subtask. Concretely, we (i) train a visual foresight predictor from images of terminal states of the base policy, labeled using downstream rollout statistics, and (ii) train residual policies via backward foresight induction, using the predictor output as a reward multiplier. On a three‑phase wrench‑based nut‑tightening assembly task in Isaac Gym (grasp, move‑insert, rotate), our method achieves 85.6% full‑task success, outperforming standard subtask residual RL (54.5%) and VLA baselines, while leaving per‑subtask success unchanged. These results highlight that improving long‑horizon performance requires shaping which successful states are produced at each sub‑task, not only whether success occurs.
Authors:Evan Sinclair Smith, Anthony Miyaguchi, Snigdha Palamari, Danté Evangelista
Abstract:
Automated individual animal re‑identification is essential for large‑scale biodiversity monitoring; however, field imagery complicates separating identity cues from nuisance variation in pose, illumination, background, resolution, and species‑specific morphology. The DS@GT ARC submission to AnimalCLEF 2026 introduces a multi‑species image‑clustering system for re‑identifying Eurasian lynx, fire salamanders, loggerhead sea turtles, and Texas horned lizards. Instead of relying on a single descriptor or nearest‑neighbor retrieval, this approach formulates re‑identification as species‑aware graph construction over candidate image pairs. The pipeline integrates tailored preprocessing, global candidate retrieval, LightGlue‑based local verification with multiple keypoint families, LightGBM pair scoring, conservative edge admission, and Leiden community detection. This design directly addresses a primary failure mode of clustering‑based re‑identification: high‑scoring false pairs that act as bridge edges and merge distinct individuals through transitive closure. Across species, ablation studies demonstrate that local feature support, foreground‑aware preprocessing, and species‑specific backbone selection enhance pair evidence, while graph operating points determine the trade‑off between fragmentation and over‑merging. The selected submission achieved a public ARI of 0.733 and a private ARI of 0.674, ranking fifth among 230 teams. These results indicate that robust wildlife re‑identification requires not only strong visual representations but also calibrated integration of global similarity, local identity markings, neighborhood context, and graph‑level constraints. The code can be found at https://github.com/dsgt‑arc/animalclef‑2026.
Authors:Heejin Jo
Abstract:
Chat models sometimes commit to an answer and then produce reasoning that justifies it rather than deriving it ‑‑ even when the answer contradicts a task premise. We study a minimal probe: "I want to wash my car. The car wash is 100 meters away. Should I walk or drive?" Only drive works (the car must be at the car wash), yet models overwhelmingly recommend walking. (1) Behavioral reproduction: on Qwen3‑8B across five system‑prompt conditions (210 rollouts), the wrong commitment occurs in 85‑100% of sampled rollouts per condition and 100% of greedy rollouts, in both thinking and non‑thinking modes; a 4,096‑token thinking budget does not repair it. (2) Preliminary activation‑level evidence: probing hidden states with a pretrained, training‑free activation oracle (no task‑specific probe training) at positions before the answer text is emitted, "walk" read‑outs exceed a neutral‑context baseline (68% vs. 17%; walk‑committing rollouts p=.005, drive‑committing rollouts p=.005, Fisher exact) ‑‑ notably, rollouts that eventually answer drive also read as walk‑leaning before commitment (5/6). The oracle's default on unrelated content is "drive" (83%), so the read‑outs are not lexical bias; stratifying by literal walk/drive occurrence shows they are not text recovery either (spans containing "drive" still read out walk; in balanced lexical fields, per‑rollout walk‑majorities beat a per‑prompt neutral baseline 15/22 vs. 1/8, p=.01; drive‑committing rollouts 6/6, p=.002). Samples are small and the within‑rollout positional gradient is not significant (p=.34); we frame these results as preliminary. (3) Methodological: with fixed oracle, activations, and positions, question wording alone moves a positive control from 2/16 (open question) to 11/16 (closed); negative oracle results are uninterpretable without per‑wording positive controls.
Authors:Sriram Balasubramanian, Soheil Feizi
Abstract:
Interpretability methods for neural network activations span a wide cost spectrum, from cheap, training‑free techniques (such as linear probes, PCA, SVD) to more expensive training‑based ones (such as SAEs and activation oracles). Training‑based methods are typically more powerful, in part because they leverage large activation datasets during training. This raises a natural question ‑ do they actually surface insights that go beyond what is recoverable from the training dataset itself? To address this, we equip an LLM agent with a vector database of activations paired with their textual contexts, along with tools for manipulating activations ‑ projecting out directions in latent space, computing activation differences and averages. The agent iteratively queries the database, forms hypotheses from the retrieved samples, and validates them by constructing linear probes. We call this method HARP, for Hypothesis‑driven Agentic Retrieval and Probing. Despite not involving any training, HARP outperforms both activation oracles and SAE‑based agents on concept discovery, concept detection, model steering, and secret elicitation. The training‑free design also makes HARP substantially cheaper and more flexible: new datasets can be indexed on demand whenever existing ones prove insufficient. More broadly, our results suggest that current training‑based methods do not yet extract insights beyond their training data, and motivate benchmarks that explicitly require interpretability methods to demonstrate such insights. We release our code at https://github.com/SriramB‑98/HARP
Authors:Pei Tian, Zihan Dong, Tianci Liu, Linjun Zhang, Haoyu Wang
Abstract:
Small‑scale language models (SLMs) are attractive for retrieval‑augmented generation (RAG) in resource‑constrained settings, but their limited capacity makes them highly sensitive to noisy or spurious retrieved evidence. Existing preference‑based methods such as RoseRAG select only the hardest single preference pair via hard argmin/argmax, discarding the remaining signal; others treat multiple pairs as independent binary comparisons, resulting in low data utilization. We propose RIMS, a three‑stage preference optimization framework comprising (1) synthetic chain‑of‑thought preference data generation via rejection sampling using the target SLM itself without relying on proprietary models, (2) a differentiable soft aggregation mechanism that replaces hard selection with a smooth operator, preserving gradient signal from all preference pairs while retaining the discriminative structure of margin‑aware selection, and (3) preference optimization with the smoothed objective applied to multiple alignment algorithms. We theoretically show that the smoothed approximation admits a controllable error bound and that smooth aggregation yields provably tighter gradient alignment to the oracle objective than hard selection. Experiments on four multi‑hop question answering benchmarks show that our approach outperforms state‑of‑the‑art baselines across multiple SLM backbones, achieving consistent gains in Exact Match and F1 under noisy retrieval conditions. Our implementation is available at https://github.com/tptrix29/RIMS.
Authors:Qirui Li, Jinkun Hao, Yibo Li, Ran Yi, Paul L. Rosin, Yu-Kun Lai
Abstract:
Recent advances in physics‑grounded video generation leverage physics simulation as a physical prior to guide video synthesis toward physically plausible outcomes. The simulation process is controlled by physical specifications, which are typically generated by a vision‑language model in a single pass. Such one‑shot prediction often fails to accurately translate user intent into executable simulations, particularly for fine‑grained object dynamics, complex motion trajectories, and temporally structured interactions. In this paper, we propose PhysAgent, a reflective agentic framework that closes the loop among physical program generation, physics simulation, stage‑specific verification, and targeted program repair. Beyond improving the control of coupled physical parameters, our framework enables the agent to progressively realize complex trajectories, multi‑stage interactions, and precise event outcomes by treating each physical program as an executable hypothesis. In addition, we design a set of physics‑control APIs to support more stable and complex motion behaviors. Extensive experiments demonstrate that PhysAgent produces more physically plausible videos, achieves better prompt alignment, and generalizes more effectively across diverse physical scenarios.
Authors:Xiaoye Zhu, Weixin Li, Junan Huo, Bozhong Wang, Jia Zeng, Yi Yang, Cen Chen, Qi Liu
Abstract:
A fundamental intent asymmetry plagues modern 3D asset creation: while state‑of‑the‑art 3D toolchains demand precise, executable parameters, ordinary users typically provide vague, underspecified instructions. Current 3D agents treat this ambiguity as noise, defaulting to blind execution under a single‑turn assumption. To address this limitation, we introduce CLARE, a clarification‑aware and evolutionary 3D agent that treats intent asymmetry not as an execution error, but as an opportunity for strategic dialogue. By decoupling the generation pipeline into four specialized cognitive roles, CLARE intercepts and resolves underspecified instructions before invoking computationally expensive 3D tools to seamlessly execute tasks across five diverse domains: text‑to‑3D generation, single‑view reconstruction, multi‑view reconstruction, point cloud editing, and post‑processing. Crucially, rather than relying on rigid manual rules, CLARE self‑evolves its clarification policy via simulated multi‑turn interactions. By optimizing a Multi‑turn Reward, the agent internalizes the delicate balance between interaction efficiency and task completion. To rigorously test this, we construct 3D‑Clarify, a comprehensive benchmark comprising 620 interaction scenarios with systematically injected ambiguity, missing information, and mistaken details. CLARE achieves state‑of‑the‑art performance, with 60.40% and 43.34% success rates on single‑step and multi‑step tasks, respectively, more than doubling existing baselines. Both quantitative and qualitative results demonstrate that proactive clarification is the missing key to robust 3D execution. Code is available at https://github.com/xyzhu1225/CLARE.
Authors:Sungjun Cho, Zhuangzhuang Chen, Xiaomeng Li
Abstract:
In this paper, we reveal an important yet overlooked problem in image denoising: under signal‑dependent camera noise models, dark regions suffer from inherently low Signal‑to‑Noise Ratio (SNR), as signal intensity decays far faster than noise variance diminishes, making detail recovery in dark areas fundamentally challenging. Yet rather than compensating for this difficulty, MSE‑trained denoisers exacerbate it ‑‑ reconstructing dark pixels up to 6x worse relative to their per‑band noise floor. This bias stems from two compounding factors: signal‑dependent noise inflates bright‑pixel residuals, and the network's Jacobian norm increases monotonically with brightness. Together, these cause bright regions to chronically dominate gradient updates at the expense of dark ones. To this end, we propose Brightness Bias‑Robust Denoising (BBRD), a drop‑in replacement for MSE loss that partitions pixels into brightness bands, normalizes per‑band error by empirical noise variance, and applies Group Distributionally Robust Optimization (Group‑DRO) to dynamically upweight whichever band is currently worst, with zero additional parameters or inference cost. Across 8 architectures and 2 datasets in our experiments, BBRD is the only method among 13 tested alternatives that improves each brightness band simultaneously, achieving up to +0.45 dB on dark bands, +0.32 dB on bright bands, and +0.65 dB aggregate Peak Signal‑to‑Noise Ratio (PSNR) on SIDD, with the largest per‑band gains in the darkest regions where detail recovery matters most. Code is available at https://github.com/xmed‑lab/BBRD
Authors:Lei Yuan, Zhongxu Hu, Jingyi Wen, Pengxing Yi
Abstract:
Cross‑domain few‑shot semantic segmentation (CD‑FSS) has predominantly been formulated as learning domain‑invariant representations or improving support‑query correspondence. Nevertheless, large domain shifts still make prototype matching unreliable: inconsistent hierarchical responses corrupt the support representation, deterministic prototypes cannot express boundary and appearance ambiguity, and treating prototypes with different reliability equally during optimization weakens foreground‑background separation. We therefore propose DAUPNet, a unified framework that reformulates cross‑domain prototype matching as uncertainty‑aware prototype discrimination. DAUPNet first harmonizes hierarchical support‑query features to provide stable evidence, then represents foreground and background prototypes probabilistically, and finally uses their estimated uncertainty to regulate contrastive optimization. On four standard target domains, DAUPNet achieves 72.6% and 76.7% average mIoU in the 1‑shot and 5‑shot settings, respectively, including substantial gains on the two medical domains. These results demonstrate that modeling prototype uncertainty and incorporating it into optimization provides a robust and interpretable approach to CD‑FSS under severe domain shift. The code is available at https://github.com/madness‑Lei/DAUPNet
Authors:Yunhang Qian, Jiaquan Yu, Jiawei Liu, Meng Wang, Hongwei Bran Li, Xiaobin Hu
Abstract:
Medical Vision‑Language Models (Med‑VLMs) require reliable reasoning from fine‑grained visual evidence, yet existing models can produce plausible clinical answers by relying on language priors or medical templates rather than truly attending to diagnosis‑critical regions. On‑Policy Distillation (OPD) offers dense token‑level supervision on student‑generated trajectories and provides a privacy‑compatible means of capability transfer without requiring the redistribution of raw patient data. However, standard OPD uniformly distills all tokens, causing sparse evidence‑dependent tokens to be diluted by abundant clinical narrative tokens. Inspired by the success of OPD in the large language model community, we propose Med‑OPD, to our knowledge the first unified post‑training framework that integrates on‑policy distillation with medical evidence‑aware supervision for Med‑VLMs. We introduce Medical Evidence Advantage (MEA), a teacher‑grounded counterfactual signal that uses an answer‑aware hint to focus teacher scoring on evidence supporting the target diagnosis, and measures each token's dependence on medical visual evidence by comparing teacher likelihoods under the original and evidence‑degraded imaging modalities. Based on MEA, Med‑OPD redistributes the distillation signal at both the token and trajectory levels, emphasizing diagnosis‑critical tokens and evidence‑reliant rollouts. Experiments on OmniMedVQA subsets show that Med‑OPD consistently outperforms SFT and standard OPD across CT, MRI, Disease Diagnosis, and Lesion Grading. These results demonstrate that evidence‑aware distillation can better strengthen medical VLMs' reliance on key visual evidence and improve reliable multimodal medical reasoning. The source code and data is publicly available at: https://github.com/yunhang8658/MedOPD.git
Authors:Tung Hung Bui, Hong Hai Nguyen, Van Thong Huynh
Abstract:
Leading entries on the multi‑task track of the 11th ABAW challenge rely on heavy ensembling, yet which member is worth adding to an already strong ensemble is rarely made explicit. We study this question for joint valence‑arousal estimation, 8‑way expression recognition, and 12‑way action‑unit detection from a single unconstrained face, under partial, long‑tailed labels and a rule that forbids pretraining on Aff‑Wild2. Building on a shared affect‑latent that marginalizes the missing labels across two affect‑supervised backbones, we propose a strength‑parity rule: an added member lowers the ensemble error only when it is both decorrelated from the current members and a near‑peer of them in individual accuracy. The rule exposes a concrete obstacle, as on a single backbone re‑seeding and even distinct fine‑tuning curricula re‑converge to a prediction correlation of 0.98 and add no diversity. Parameter‑isolation removes it: confining each adaptation to a disjoint low‑rank subspace of a shared backbone yields experts that stay decorrelated at 0.91 while remaining near‑peers, the strongest of them an AffectNet‑adapted expert. The resulting system raises the overall validation score to 1.6949, against the organizers ConvNeXt‑with‑MixAugment baseline of 0.45; with per‑AU calibration and by pooling the shared‑latent heads valence‑arousal byproduct as a further near‑peer, the strongest configuration reaches 1.7259. Source code are available at https://github.com/cprl‑team/MTL‑ABAW‑11th.
Authors:Yingzhao Jian, Zihao Lin, Hehe Fan
Abstract:
The 3D geometry of real‑world scene data is often incomplete. Mainstream methods use depth estimators to inpaint missing structure. However, their prediction results can be inconsistent with observed geometry, or unreliable on out‑of‑distribution data. To solve these problems, we propose Neural Depth Field (NDF). Our key insight is that a depth estimator can also be a scene‑level implicit field. As an estimator, it adapts to the target domain by learning observed depth data. As an implicit field, it fits the existing geometry to maintain consistency. Under this view, NDF addresses both problems through a single test‑time optimization. Experiments show that NDF produces high‑fidelity and globally consistent geometry across diverse scene data, ranging from indoor scans to satellite imagery. It reduces cross‑view inconsistency by 63.3% and improves inpainting accuracy by 23.1%, achieving state‑of‑the‑art performance in 3D scene geometry inpainting. The code is available at: https://github.com/Shadow‑Dream/Neural‑Depth‑Field.
Authors:Muhammad Emmad Siddiqui, Muhammad Rafi
Abstract:
Facial image retrieval in unconstrained surveillance environments is a high‑stakes challenge where missing a subject of interest ‑‑ a single false negative ‑‑ is simply not an option. Despite near‑perfect performance on curated benchmarks, current recognition systems falter under real‑world domain shifts such as low resolution, motion blur, and uncontrolled illumination (e.g., SCFace). Addressing this reliability gap, we propose Risk‑Aware Facial Retrieval (RA‑FR), a framework that moves beyond fixed Top‑k retrieval to adaptive set generation, guaranteeing ground truth inclusion within a user‑specified risk level (α) and confidence level (1 ‑ δ). Our approach integrates three core contributions: (1) reducing aleatoric uncertainty via a hybrid blind face restoration technique coupling Latent Consistency Models (InterLCM) and DiffBIR; (2) extracting discriminative, restoration‑robust features via self‑supervised DINOv1 ViT‑B with GGeM pooling; and (3) employing conformal prediction with Hoeffding's inequality to dynamically calibrate retrieval set sizes based on query uncertainty. On the IMFDB benchmark, it consistently satisfies a 5% risk target with an average retrieval set size of approximately 10 images. By unifying domain‑specific restoration, robust representation learning, and provable decision rules, RA‑FR offers a pipeline that makes facial retrieval in surveillance both reliable and auditable. The code is available at: https://github.com/MuhammadEmmadSiddiqui/RA‑FR.
Authors:Ali Sultonov
Abstract:
Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence. We introduce PsiLogic, an optimizer that augments Adam with a dynamic Active Cancellation Term gated by a dual exponential moving average (EMA) of scale‑normalized gradient norms. The resulting chaos detector strengthens damping when gradient statistics are unstable and fades to zero as training stabilizes, providing an implicit warmup without a hand‑tuned schedule. We evaluate PsiLogic against Adam, AdamW, and Lion using FairBench ‑‑ a reproducible benchmark protocol with per‑optimizer learning‑rate sweeps, identical initialization per seed, and Welch t‑tests. On an NVIDIA H100 80GB reference run (4 arenas, 3 seeds, 2000 steps, bf16 AMP), PsiLogic achieves the best validation metric in three of four arenas: NLP perplexity 7.79 +/‑ 0.18 vs. 8.17 +/‑ 0.08 (AdamW, p = 0.049), ViT top‑1 accuracy 0.244 +/‑ 0.006 vs. 0.223 +/‑ 0.002 (AdamW, p = 0.015), and ResNet top‑1 accuracy 0.222 +/‑ 0.001 vs. 0.172 +/‑ 0.004 (Adam, p = 0.001). On diffusion, validation MSE is statistically tied with Adam/AdamW (p = 0.49). ResNet accuracy vs. AdamW is a numerical tie without significance at three seeds (p = 0.44). Peak GPU memory is comparable across optimizers; PsiLogic incurs 1.2‑‑1.8x wall‑clock overhead on transformer‑heavy arenas (implementation‑bound). We release an open‑source PyTorch implementation, the full FairBench harness, and all raw CSV outputs to support independent verification.
Authors:Josh Qixuan Sun, Morteza Babaie, Wenyang Hou, Mark Crowley, David Young
Abstract:
Antibody expression ranking is a critical task in antibody design, yet its modelling is severely hindered by the scarcity of labeled experimental data. To address this, we propose a unified preference‑based learning framework that integrates scarce quantitative expression data with large‑scale weak positive supervision from immunization data. We adapt Direct Preference Optimization (DPO) to protein language models by introducing a union‑masked log‑likelihood approximation and IMGT‑based alignment, enabling efficient training on variable‑length sequences. Evaluating on a diverse internal dataset of 1254 labeled sequences and 4 million unlabeled camelid‑derived antibodies, we show that our method consistently outperforms baselines on most metrics. Our results demonstrate that preference learning can effectively learn from weak supervision, providing a scalable solution for antibody expressibility optimization in data‑constrained settings. Project page: https://kisoji‑biotechnology‑inc.github.io/Preference‑Expression‑Ranking/.
Authors:Jialong Zhong, Tingwei Liu, Baokun Yue, Jingjing Li, Yongri Piao, Miao Zhang, Leiye Liu, Jiahong Jiang, Wei Ji, Huchuan Lu
Abstract:
Multimodal survival analysis utilizing whole slide images (WSIs) and genomic profiles is fundamental for cancer prognosis. Recently, state‑space models like Mamba have emerged as powerful tools for sequence modeling. However, translating this success to complex multimodal tasks is hindered by two critical limitations. First, conventional fusion strategies assume a static multimodal interaction strength, ignoring the fluctuating diagnostic importance of each modality across different patients and local regions. Second, the standard Mamba architecture processes tokens along predefined physical paths. This rigid scanning disrupts the semantic continuity of spatially scattered medical features and exacerbates long‑range decay. To address these challenges, we introduce AdaSurvMamba as a novel adaptive framework for multimodal survival analysis. The framework features a Dual‑Scale Importance‑Aware Reconstruction (DSIR) module to dynamically modulate cross‑modal interaction strength. It evaluates diagnostic importance at both the sequence and token levels to reconstruct the input representations. Furthermore, we propose a Semantic Aggregation Scanning (SAS) module to overcome contextual fragmentation. The SAS module dynamically reorganizes discrete tokens into semantically continuous sequences via a shared prototype pool. It explicitly modulates the state transition step size using global modality context and semantic priors to adaptively control the information absorption rate. Experiments across five TCGA cohorts demonstrate consistent gains over existing methods. Code is available at https://github.com/zjlGO/AdaSurvMamba.
Authors:Yupeng Chang, Yuan Wu, Yi Chang
Abstract:
Low‑Rank Adaptation (LoRA) is a widely used parameter‑efficient fine‑tuning (PEFT) method for large language models. Under a fixed rank budget, LoRA parameterizes each adapted weight through a single low‑dimensional input‑side pathway, which may couple heterogeneous behaviors through shared input directions and induce interference during optimization. We propose Static Orthogonal Subspace LoRA (SOS‑LoRA), a drop‑in extension that reparameterizes a rank‑rtot update as a sum of K static (always‑on, non‑routed) low‑rank experts. SOS‑LoRA (i) decomposes the total rank across experts, (ii) applies a fixed multi‑scale scaling scheme to encourage scale‑separated optimization dynamics, and (iii) promotes diverse input‑side directions via cross‑expert orthogonal initialization and a lightweight regularizer. SOS‑LoRA remains fully mergeable, adding no inference‑time parameters or latency after merging. Experiments on reasoning and knowledge‑intensive benchmarks (Llama 2/3), encoder‑based NLU (GLUE), and math reasoning (GSM8K/MATH) show consistent gains over matched‑budget LoRA baselines and recent variants. Code is available at https://github.com/llm172/sos‑lora.
Authors:Bingrui Sima, Lizhong Wang, Xiaoya Lu, Kun He, Xiao Yang
Abstract:
While Vision‑Language Models (VLMs) have empowered embodied agents to execute complex household tasks, they struggle to proactively handle dynamically emerging hazards during closed‑loop interactions. Existing safety approaches often rely on runtime guardrails to block unsafe actions or induce excessive caution, which severely stalls task progress instead of actively resolving the underlying risks. To break this safety‑progress trade‑off, we introduce the Self‑Evolving Just‑In‑Time Memory framework, which reframes embodied safety from progress‑stalling guardrails to proactive hazard mitigation. The framework consists of a Risk‑Sufficient Topological Belief Graph (RSG) for persistent safety‑relevant state tracking under partial observability, an Agency‑Grounded Factual Memory for precise hazard anticipation, and an Experience Memory that injects procedural Meta‑Skills to guide executable, progress‑preserving mitigation. Furthermore, we propose an automated Test‑Verify‑Write loop, allowing agents to continually refine their mitigation Meta‑Skills from execution traces at test time. Experiments on IS‑Bench demonstrate that our framework substantially boosts the Safe‑Success rate across multiple VLM backbones (e.g., +30.3% on Qwen3‑VL‑8B), enabling agents to proactively mitigate hazards without stalling task progress. Code is available at https://github.com/DyMessi/JIT‑Memory.
Authors:Anushiya Arunan, Xin Li, Yan Qin, U-Xuan Tan, Nhu Khue Vuong, Xiaoli Li, Chau Yuen
Abstract:
Multimodal industrial anomaly inspection assistants are a critical component of next‑generation smart factories, enabling interactive vision‑language‑based querying. However, multimodal large language models remain impractical for on‑site deployment due to prohibitive computational demands and privacy risks from cloud‑based inference. Compact multimodal small language models (MSLMs) offer a deployable alternative, yet progress is constrained by the lack of comprehensive robustness analyses and meaningfully challenging benchmarks that reflect real‑world industrial conditions. To address this gap, we develop RobustMAD, the first deployment‑motivated benchmark, designed to comprehensively evaluate model robustness through diverse open‑ended queries spanning object understanding, anomaly detection, unanswerable problems, and visual quality degradations. Contrary to conventional assumptions, top‑performing MSLMs exhibit promising capabilities, surprisingly outperforming even the larger GPT‑5 Nano. However, they still fall short of safety‑critical requirements, and RobustMAD reveals critical robustness gaps that pose operational risks. In particular, three recurring failure modes emerge: (i) fragile multimodal grounding under fine‑grained distinctions or degraded visual conditions, (ii) insufficiently comprehensive responses, and (iii) weak logical grounding on unanswerable or ill‑posed queries, leading to hallucinated outputs. Grounded in these insights, we provide actionable guidance for the design of next‑generation multimodal industrial inspection assistants that leverage their promising competence. Code is available at https://github.com/en‑research/RobustMAD.
Authors:Sumit Verma, Pritam Prasun, Pritish Kumar
Abstract:
Existing guardrail systems for large language model agents operate as binary classifiers that block unsafe content, leaving organizations to discard failing outputs and retry from scratch. We introduce RAIL Guard, a closed‑loop responsible AI pipeline that evaluates LLM outputs across eight measurable dimensions and iteratively remediates failing outputs through an evaluate‑rewrite‑reevaluate loop. We evaluate the pipeline across three experiments on four frontier LLMs and 4,276 content outputs plus 6,400 agent tool‑call scenarios. Closed‑loop remediation achieves 96.9% convergence versus 49.1% for block‑and‑retry, though the highest‑convergence method reduces utility by 22.3%; feedback‑driven self‑repair achieves 86.6% convergence on fixable dimensions with no significant utility loss (p = 0.177). Pre‑tool‑call evaluation reduces unsafe agent executions by 33% (p = 0.007) with zero impact on task completion. We identify a key distinction between fixable dimensions that respond to remediation and structural dimensions (Transparency at 93.0%, Accountability at 92.8%, and Inclusivity at 82.5% failure) that require architectural rather than algorithmic solutions. The system is available as open‑source SDKs.
Authors:Darshan Deshpande
Abstract:
Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments. Hand‑curated environments with fixed task and reward difficulties become ineffective signals as model performance improves, and sparse rewards over long horizons induce mode collapse on specific workflows or tool structures. World models that simulate environment states have matched pure rollout performance, making them promising for scaling diversity on‑demand. However, autoregressive (AR) world models suffer from a left‑to‑right bias preventing conditioning on globally interdependent state anchors such as tool schemas, prior turns, and expected outcomes. We (i) formalize text‑based world modeling as a steerable transition‑dynamics problem decomposed into initial state, task context, tool schemas, domain rules, and steering directives, and (ii) curate 239,403 grounded state‑action trajectories spanning nine open‑source environments and twelve frontier model families. We compare AR LMs and masked diffusion language models (MDLMs), showing MDLMs, via bidirectional anchor‑aware denoising, achieve better coherence, groundedness, and empirically validated rollout diversity than LLMs over 4x their parameter size, at comparable inference latency. We introduce a plug‑and‑play GRPO training framework with deterministic state checks, and perform zero‑shot transfer ablations on three OOD environments (ScienceWorld, ALFWorld, AppWorld) across three 1.2B‑7B agent backbones (LFM2.5, Qwen3, Mistral), achieving up to 47% absolute gains over baselines without environment‑specific fine‑tuning. We further conduct behavioral analysis of failure modes under adversarial scenarios and human evaluation on realism, outcome correctness, and training utility. We open‑source our work to encourage research in this direction.
Authors:Chengcheng Sun, Yajie Song, Cheng Zhai, Jiayun Tian, Jia Yang, Xiaobin Rui, Jian Zhang, Zhixiao Wang, Philip S. Yu
Abstract:
Graph Neural Networks (GNNs) have emerged as the leading paradigm for link prediction, enabling the inference of missing connections and the anticipation of potential future links. However, existing reviews lack systematic exploration specifically targeting underlying GNN architectures and diverse graph structures. To address this critical gap, this paper provides a comprehensive review of GNN‑based link prediction from a novel and dedicated GNN perspective. We propose an innovative taxonomy that categorizes recent advancements based on techniques and applications. From a technique perspective, we focus on key GNN encoder architectures, including GCN‑based, GAE‑based, GAT‑based, and GFormer‑based methods, discussing their strengths and limitations. From an application perspective, we highlight prominent use cases of link prediction in knowledge graphs and recommendation systems, demonstrating their real‑world impact. In addition, we examine the current challenges and discuss promising future directions.
Authors:Ce Zhang, Ziyang Wang, Yulu Pan, Oluwatumininu Oguntola, Pranav Wagh, Qiyu Wu, Hiromi Wakaki, Mohit Bansal, Gedas Bertasius
Abstract:
Grounded long‑video question answering (Grounded LVQA) requires answering a question about a long video while localizing the short evidence interval that supports the answer. Recent agentic methods frame this task as multi‑turn exploration with a single crop_video(start, end) action, which supports coarse‑to‑fine narrowing but provides no primitive for fine‑to‑coarse backtracking. As a result, these agents typically converge prematurely and cannot recover from an early mistake. We propose VideoTreeSearch (VTS), a framework that casts grounded LVQA as iterative self‑correcting search over an adaptive temporal tree. VTS constructs a non‑uniform tree from visual scene boundaries so that each node corresponds to a semantically coherent segment, and trains an agent to navigate the tree through four discrete operations: zoom_in, zoom_out, shift, and answer. These operations expose backtracking and recovery as explicit, learnable primitives rather than implicit behaviors. To train this navigation, we introduce a trajectory synthesis pipeline that produces multi‑step paths through the tree, including deliberate detours into incorrect branches followed by recovery. We use these trajectories for supervised fine‑tuning, followed by reinforcement learning with grounding and answer‑accuracy rewards. On three Grounded LVQA benchmarks (CG‑Bench, Haystack‑LVBench, Haystack‑Ego4D), VTS outperforms the strongest prior agentic methods by +12.5 mIoU on CG‑Bench and +7.4 T‑F1 on Haystack‑Ego4D. The learned policy also transfers to general long‑video QA, surpassing all prior agentic baselines on Video‑MME, MLVU, and LVBench by up to +7.1 accuracy points. Ablations confirm that self‑correcting hierarchical search is the central mechanism behind these gains: removing either adaptive descent or explicit backtracking substantially degrades performance. Code is available at https://github.com/CeeZh/VTS.
Authors:Kai Ruan, Jinghao Lin, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun
Abstract:
Muon is competitive with AdamW in large‑scale pre‑training, but its operating regime in reinforcement‑learning post‑training remains unclear. We map this regime on ALFWorld, a sparse‑reward agentic benchmark, using three group‑based objectives and Qwen2.5 models from 0.5B to 3B. Under a shared KL and clipping recipe, matched optimizer comparisons and AdamW rate controls trace the usable step‑size range. AdamW responds non‑monotonically to rate, whereas fan‑in Muon remains stable at a more aggressive effective step: at 3 × 10^‑5 it improves late success over an AdamW 10^‑6 baseline after correction across rate‑metric tests. Its normalized‑AUC effect is directionally positive but less uniform; the heuristic‑matched lower‑rate effect is less consistent, and tuned AdamW nearly matches high‑rate Muon at 3B GraphGPO. High‑rate Muon applies 3.53 × AdamW's hidden‑matrix update RMS; a full‑budget RMS‑matched control removes the late‑success gain. Together, these results identify a recipe‑level operating regime in which fan‑in Muon supports a more aggressive stable effective step under shared KL and clipping: the margin is largest when optimization headroom remains and contracts near saturation, after AdamW tuning, or under magnitude matching. The scale‑matched control ties this spectral effect to Muon's scale convention rather than establishing a universal optimizer ranking. Code is available at https://github.com/x66ccff/verl‑muon.
Authors:Ingo Ziegler, Martin Krebs, Desmond Elliott
Abstract:
Language models encode text as subword tokens, raw bytes, or rendered pixels, but these encodings are usually compared under modeling constraints that expose different amounts of linguistic content to models across different languages. We instead ask what each encoding preserves when both the content and the downstream capacity are controlled. Using verified parallel sentences across thirteen languages and five scripts, we compare tokens, bytes, and pixels through a shared bottleneck whose width is swept to trace rate‑utility frontiers. This separates three quantities that are often conflated: the number of input positions an encoding creates, the latent capacity available after encoding, and the task‑relevant information that survives compression. We evaluate three utilities: surface form preservation, cross‑lingual sentence alignment, and topic classification. No encoding dominates across tasks or capacity regimes. Pixels preserve surface form best, bytes preserve cross‑lingual alignment best, especially in same‑script multilingual settings, and tokens support topic prediction best. These performances are not explained by sequence length alone. Short inputs can discard useful meaning, while long inputs can preserve information that compresses well. Choosing an encoding is therefore not a fixed preference for tokens, bytes, or pixels, but a rate‑utility tradeoff that depends on the task, language mix, capacity regime, and compute budget.
Authors:Hanyang Chen, Anirudh Satheesh, Longchao Da, Hua Wei
Abstract:
Transferring policies across domains poses a vital challenge in reinforcement learning, due to the dynamics mismatch between the source and target domains. In this paper, we consider the setting of online dynamics adaptation, where policies are trained in the source domain with sufficient data, while only limited interactions with the target domain are allowed. There are a few existing works that address the dynamics mismatch by employing domain classifiers, value‑guided data filtering, or representation learning. Instead, we study the domain adaptation problem from a generative modeling perspective. Specifically, we introduce DADiff, a diffusion‑based framework that leverages the discrepancy between source and target domain generative trajectories in the generation process of the next state to estimate the dynamics mismatch. Both reward modification and data selection variants are developed to adapt the policy to the target domain. We also provide a theoretical analysis to show that the performance difference of a given policy between the two domains is bounded by the generative trajectory deviation. More discussions on the applicability of the variants and the connection between our theoretical analysis and the prior work are further provided. We conduct extensive experiments in environments with various shifts to validate the effectiveness of our method. The results demonstrate that our method provides superior performance compared to existing approaches, effectively addressing the dynamics mismatch. We provide the code of our method at https://github.com/hanyang‑chen/DADiff‑release
Authors:SciForge Team, Zhangyang Gao, Minghao Fang, Yifei Liu, Hanhui Yang, Xinyu Gu, Shixiang Tang, Siqi Sun, Lei Bai, Cheng Tan, Mengdi Liu, Hao Wu, Shuizhou Chen
Abstract:
Scientific work increasingly spans heterogeneous artifacts ‑‑ papers, code, datasets, scientific file formats, model outputs, figures, manuscripts, and team decisions ‑‑ yet general‑purpose AI assistants rarely preserve these objects as a coherent, auditable research state. We present SciForge, a multimodal research‑native AI workbench that reserves the graphical interface for human judgment while search, parsing, model routing, workflow execution, plotting, writing, and presentation generation run as modular agent‑accessible services. SciForge is built around five pillars: (i) \emphgoal‑scoped scientific decision governance for goal‑oriented research, with review gates and shared review surfaces; (ii) \emphtranslate‑then‑reason for multimodal input, routing scientific objects through domain translators before the agent reasons; (iii) \emphevidence governance for auditable traceability, linking claims to provenance chains and audit findings; (iv) \emphcollaborative team science for collaborative research, enabling multi‑role decision governance, with shared team workspaces planned for future releases; and (v) \emphreal‑world application scenarios for practical impact, demonstrated through eight end‑to‑end user cases, with flagship demonstrations including multi‑day agentic research sprints for gene discovery, AI‑guided de novo protein design, molecular optimization, and genome‑to‑BGC discovery. The system combines a thin interaction layer, contextual research capability patterns, an Agent Runtime and Workflow Engine, an Evidence‑DAG audit sidecar and a Scientific Model Router. SciForge currently runs as a desktop application, with mobile supervision support; future releases will deepen team collaboration. The system is open‑source and available at https://github.com/AGI4Sci/SciForge
Authors:Jehun Kang, Jungha Wang, Youngjun Hwang, David Hyunchul Shim
Abstract:
Multi‑Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation. While Vision Foundation Models (VFMs) are increasingly adopted as robust feature encoders, existing decoding strategies present a critical bottleneck. To address this, we propose DPNeXt, a streamlined multi‑scale feature fusion decoder and efficient alternative to the standard Dense Prediction Transformer (DPT). DPNeXt uses dual depthwise separable inverted bottlenecks to improve frozen VFM utilization through fusion‑centric decoding and independent task modularization. To further mitigate negative inductive transfer between tasks, we introduce the Multi‑Task Boundary Guidance (MTBG) strategy. Unlike prior boundary‑aware methods that add fusion modules or gating, MTBG applies symmetric boundary‑focused supervision to encourage geometric consistency without extra annotation or inference cost. Experiments on Cityscapes show that DPNeXt‑S outperforms prior state‑of‑the‑art (SOTA) MTL models, while DPNeXt‑B further improves the overall performance and achieves the best results among the compared methods. On NYUv2, DPNeXt‑B also achieves the best semantic segmentation and depth estimation results among the compared methods while requiring substantially fewer trainable parameters than prior large‑scale MTL models. Compared with the standard DPT, DPNeXt‑S reduces trainable parameters by 78.6% and achieves the fastest inference speed among the compared models on resource‑constrained laptop hardware. The source code, model checkpoints, and a demo video will be made available at https://github.com/kangjehun/DPNeXt.
Authors:Hui Wei, Seyedata Jodeiri Seyedian, Xiaobai Li, Guoying Zhao
Abstract:
Deep remote photoplethysmography (rPPG) attains sub‑bpm heart‑rate error on frontal, stationary faces yet degrades sharply under head pose: on MMPD, the state‑of‑the‑art FactorizePhys backbone's MAE grows 1.60× from frontal (|\textyaw|<15^\circ) to large‑yaw (|\textyaw|\geq45^\circ) frames. We argue that pose is a \emphcoordinate‑structural nuisance rather than a data‑augmentation problem: in image coordinates the same pixel maps to different anatomy at different poses, blocking three priors otherwise natural for rPPG, namely the dichromatic reflection model, pulse‑phase invariance across skin regions, and the POS/CHROM chromaticity projection, each of which presumes a stable anatomy‑to‑pixel mapping. We introduce CanonicalPhys, which prepends a differentiable four‑point homography that fixes four facial anchors at canonical positions; in this canonical frame the three priors become expressible as a per‑pixel Lambertian weight, a cross‑ROI temporal consistency loss, and knowledge distillation from windowed POS, none of which adds trainable parameters over the backbone. At an identical parameter count, CanonicalPhys reduces MMPD's frontal‑to‑large‑yaw MAE degradation from 1.60× to 1.33× and flattens the mild‑yaw bin from 1.32× to 1.07× (across CanonicalPhys variants), with matched cross‑dataset MAE reductions of up to 32% on pose‑rich targets. Code: https://github.com/infraface/CanonicalPhys
Authors:Théophane Loloum, Fabien Vivodtzev, David Hébert, Baptiste Reynier, Michel Arrigoni, Julien Tierny
Abstract:
This application paper presents DebrisTracer, a framework for the reliable tracking of debris in hypervelocity impact fast imaging. These noisy and highly specific datasets capture the ejection of a large number of debris fragments after the impact of a projectile launched at hypervelocity into a target material. The reliable estimation of debris mass and speed distributions is of major importance in aerospace applications. We document how to extend an off‑the‑shelf topology tracking framework based on critical point extraction and matching, in order to incorporate domain knowledge and physical assumptions. Our approach automatically produces an accurate and reliable debris tracking, enabling an interpretable visual analysis of this complex space‑time phenomenon. Extensive experiments demonstrate the accuracy improvements provided by our approach over established tools used by domain experts in terms of physical validation, specifically via the prediction of the experimental ejected mass and crater depth profiles. We illustrate the utility of our approach across several use cases (with varying impact angles and physics). We show that our statistical summaries enable the visual identification of distinct regimes within the debris population, corroborating and refining prior expectations of domain experts. Our database and our C++ implementation are available at this address: https://github.com/tloloum/DebrisTracer.
Authors:Hadrien Crassous, Mohamed Yassine Kabouri, Minahil Raza, Joni Pajarinen, Riad Akrour
Abstract:
This paper studies how to adapt a computer vision object detector to an unknown environment under both a robot navigation time and annotation budget constraint. Our approach selects informative robot trajectories and image samples to retrain the detector, explicitly targeting its failure cases. Formally, the approach is an embodied variant of batch active learning, where at each round an agent has a limited navigation budget to collect candidate samples and a limited annotation budget for the most relevant images. We leverage spatial consistency to identify images with inconsistent labels, which are likely to provide the greatest improvement to the vision model. We evaluate the approach using different active learning objectives on large scenes from the AI2‑THOR simulator and on a real‑world setup using a Boston Dynamics Spot robot with the real‑time object detector YOLOv5. Through comparison against several baselines, our experimental results show that spatial inconsistency helps guide the agent and select relevant images without external supervision, achieving the highest detection accuracy at the end of the adaptation process under the same budget. The open‑source project can be found at https://mkabouri.github.io/embodied‑active‑learning‑od
Authors:Nicholas Fry, Ignacio Alzugaray, Mark Pupilli, Paul H. J. Kelly, Andrew J. Davison
Abstract:
We present the first implementation of a 3D Gaussian renderer on an Intelligence Processing Unit (IPU), comprising 1,472 independent tiles with only on‑chip SRAM; constraints that approximate properties of efficient sensor‑processor architectures. Our input scenes are 3D Gaussian maps from real‑world sequences. Each tile 'owns' a screen‑space region of the framebuffer; Gaussian primitives are routed to destination tiles via Manhattan‑distance hops on a north‑east‑west‑south (NEWS) grid, then distributed to overlapping neighbours in an expanding tree pattern. Computation follows the IPU's Bulk Synchronous Parallel (BSP) model, with inter‑tile communication defined at compile time. We show this hardware allows us to exploit spatial and temporal locality by enabling local data transfer between cores. We evaluate the bottlenecks in this SRAM‑only implementation: inter‑tile bandwidth, per‑tile SRAM capacity, and workload imbalance from non‑uniform Gaussian density. We analyse how these constraints affect performance and render quality. This exploration raises broader questions for conventional GPUs and 3D representations, suggesting that direct inter‑SM (streaming multiprocessor) communication might offer ways to reduce DRAM access in GPU kernels. We discuss these implications for the future of on‑sensor and DRAM‑free architectures. Project page: https://nmjfry.github.io/ipu‑3dgs/
Authors:Vishal Pandey, Gopal Singh
Abstract:
In production large language model (LLM) deployments, high API availability guarantees do not equate to conversational continuity. When a primary provider experiences an outage or strict rate‑limiting, naive stateless failover mechanisms successfully maintain uptime but silently discard conversation history, severely disrupting the user experience. To rigorously quantify and resolve this failure mode, we introduce two novel metrics: Continuity Preservation Rate (CPR) and Continuity Latency Overhead (CLO). We propose a stateful, multi‑provider proxy architecture utilizing a History‑Forwarding strategy to seamlessly reconstruct conversational state across heterogeneous LLM endpoints during failover events. Furthermore, we release continuity‑bench, https://github.com/Vishal‑sys‑code/continuity‑bench, an open evaluation harness designed to stress‑test context preservation under high‑concurrency provider failure conditions. Our empirical evaluation (N=750 failover events) demonstrates that our stateful proxy achieves a 99.20% CPR [95% CI: 98.27%, 99.63%], cleanly transferring deep conversational context to fallback providers, compared to a near‑0% preservation rate for standard stateless architectures. Finally, we characterize failover latency distributions, identifying the critical necessity of asynchronous exponential backoff with jitter to prevent cascading retry storms against strict‑limit fallback APIs. Our results provide a principled foundation for building robust, state‑preserving multi‑model inference systems.
Authors:Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso, Karim Farid, Jonannes Dienert, Rajat Sahay, Thomas Brox
Abstract:
Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required for real‑world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high‑level predictor that forecasts coarse scene structure over extended temporal horizons, and a low‑level generator that produces detailed predictions conditioned on the high‑level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations. We further show that pretraining with a diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing ‑‑ predicting only the next frame from clean context ‑‑ produces more stable autoregressive rollouts. We therefore introduce a generic two‑stage training paradigm that pretrains the model with diffusion forcing and fine‑tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter. Our approach achieves state‑of‑the‑art results across the standard suite of driving world model evaluations on established benchmarks, including long‑horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality. Project page with code, demo, checkpoints and qualitative results: https://lmb‑freiburg.github.io/orbis2.github.io/
Authors:Shiva Agrawal, Savankumar Bhanderi, Zhiran Yan, Gordon Elger
Abstract:
Accurate temporal alignment of heterogeneous sensors is necessary for reliable environment perception in roadside multi‑lidar, multi‑camera systems, particularly in dense urban traffic. For this purpose, an open‑source, simple, modular, and configurable hardware‑triggered time‑synchronization circuit is presented in this work to perform temporal alignment or accurate time synchronization between a lidar and multiple cameras. In the designed circuit, a lidar synchronization pulse is used as a reference input, and independently programmable, time‑delayed trigger pulses are generated for each camera, allowing flexible adaptation to varying sensor setups and mounting geometries. A series of experiments is conducted on a roadside‑mounted perception system comprised of lidar and three cameras, in which the trigger delay is systematically varied, and its impact on spatial‑temporal alignment is evaluated. For different classes of road users, the overlap between lidar point cloud measurements and camera measurements is quantified to identify delay configurations that maximize cross‑sensor consistency. The proposed circuit is shown to achieve robust and repeatable synchronization while remaining straightforward to deploy, reconfigure, and extend due to its simple and open‑source design. Following validation on a three‑camera roadside system, the circuit is extended to a vehicle platform with seven cameras and a lidar, providing a low‑cost, extensible solution for multi‑sensor synchronization across infrastructure and vehicle setups. All hardware circuit design files and source codes are available at https://github.com/shiva‑THI/hardware‑trigger‑time‑sync‑lidar‑cameras.
Authors:Haram Choi
Abstract:
Human label variation in natural language inference is increasingly treated as signal rather than noise, but how much of it formal semantic structure explains has not been measured directly. We measure it on the 3,113 SNLI and MNLI items of ChaosNLI, using a rule‑based operator and monotonicity tagger validated against MED (0.883 agreement at the edit site, 0.807 on the sentence‑level summary our analyses consume), three preregistered analysis blocks, and full reporting of negative results. Three bounds emerge. First, a group‑level boundary: hypotheses that are not purely upward monotone show reliably higher label entropy (Cliff's delta = ‑0.284), and rank‑based tests defend the effect against operator‑presence and length reductions, though a bounded‑outcome sensitivity check weakens the regression form of the length defense. Second, an item‑level ceiling: the same formal profiles explain only 3.3 to 3.6 percent of entropy variance and reach a median‑split AUC of 0.606, too weak to identify high‑disagreement items. Third, composition invariance: across the boundary, three high‑powered preregistered contrasts on validated error shares and explanation‑type shares (VariErr, LiTEx) all return null results. In this sample, formal semantic structure shifts how much annotators disagree by a small amount and does not detectably change what they disagree about. ChaosNLI‑S/M consists of items selected for low original agreement, and every claim is conditioned on that scope. All analyses were preregistered in a version‑controlled research log, whose audit trail, including one corrected interpretation rule, the paper discloses.
Authors:Pengchao Hu, Zhibin Xin, Yifan Chen, Yangyang Zhou, Liang Wang, Xin Zhang
Abstract:
Large Language Models (LLMs) have become the dominant workload on modern AI accelerators, yet deploying them on specialized hardware still faces two core challenges: how to import a trained model into a compiler‑friendly intermediate representation, and how to efficiently schedule the autoregressive inference loop under limited on‑chip memory. This paper presents an MLIR (Multi‑Level Intermediate Representation) based compilation method for large language models, illustrated using two dialects of operators, TopOp and TpuOp. TopOp serves as a high‑level graph dialect that is independent of both the source framework and the target chip, and is responsible for expressing model semantics; TpuOp serves as the target hardware dialect, carrying chip‑related decisions such as quantization, layer groups, and memory layout. A model is first represented as TopOp, then lowered layer by layer to TpuOp, and finally a deployable binary is generated. In addition, each Transformer layer is split into three stages for static compilation: prefill, prefill_kv (prefill with historical key‑value cache), and decode, so as to accommodate the different computational characteristics of prompt‑parallel processing and per‑token generation. The method has been implemented in the TPU‑MLIR compiler https://github.com/sophgo/tpu‑mlir and the LLM‑TPU deployment project https://github.com/sophgo/LLM‑TPU, supporting a variety of generative models including the Qwen, Llama, InternVL, and MiniCPM‑V series, as well as multiple quantization and deployment forms such as GPTQ, AWQ, and AutoRound.
Authors:Muness Castle, Eric Rubeck
Abstract:
Coding agents can fix a failing example without preserving the domain rule that made it fail, so later generations can repeat the same plausible mistake. We present agentic synthesis against counterexample‑supplemented sketches, a repository‑native method for systems whose governing policy is discovered during implementation. A human starts with a partial, code‑shaped sketch, and a coding agent generates the first implementation. When a concrete failure exposes missing or mistaken policy, an operator explicitly approves the corrected behavior and rule. The agent then revises the sketch and repairs or regenerates code and prompt surfaces for that one counterexample. The full archive preserves provenance; a selected regression set gates each revision before the next candidate is revealed; and periodic clean regeneration tests whether the evolved sketch, rather than prompt history or accumulated examples, carries the learned policy. We demonstrate the method with CatSynth, a synthetic browser application and captured coding‑agent experiment. In one open‑world run with GPT‑5.4‑mini, 8 of 14 frozen candidate cases became counterexamples. The rebuild controls inherited that promotion schedule, and all three paths passed the 8 accepted cases. Rebuilding from the evolved sketch passed 19 of 21 withheld cases, compared with 15 of 21 when rebuilding from the initial sketch and replaying all accepted examples. Retaining code across counterexamples required 9 Developer calls and 719 lines of cumulative artifact churn, versus 15 calls and 2,394 lines for replay‑all, and passed 18 of 21 withheld cases. These results provide inspectable evidence that the evolved sketch carried reviewed policy and that retaining code reduced rework in this run; with one model and one reveal order, they do not establish general superiority or correctness beyond the encoded checks.
Authors:Jiazhen Huang, Zhiming Liu, Changhu Wang, Wei Ju, Ziyue Qiao, Xiao Luo
Abstract:
A range of methods aim to enhance the performance of vision‑language models (VLMs) at test time. Among them, transduction has emerged as a promising paradigm due to its strong compatibility and efficiency. However, realistic evaluations often involve highly imbalanced class distributions, which cause performance degradation or even collapse. In this work, we systematically revisit transduction from the perspective of penalized likelihood estimation (PLE), showing that PLE with a KL‑divergence anchor term naturally yields an adaptive shrinkage behavior between prior anchors and empirical estimates. From this viewpoint, the brittleness of transductive methods can be attributed to the absence of anchoring mechanism and static modeling of the shrinkage strength. Therefore, we propose Mixture of Von Mises‑Fisher Models with Dynamic Shrinkage (MOON). MOON is built upon a mixture of von Mises‑Fisher distributions to model feature representations on the unit hypersphere. To handle imbalance, MOON dynamically adjusts the shrinkage strength using zero‑shot priors at both instance and class levels. Thus, it suppresses unreliable assignments and prevents harmful updates from outlier classes, thereby mitigating negative transfer. MOON is model‑agnostic, training‑free, and requires no task‑specific hyperparameter tuning. Extensive experiments further validate the advantage of MOON in both performance and efficiency. Our code is available at https://github.com/walawalagoose/MOON
Authors:Dimitrios Karageorgiou, Symeon Papadopoulos, Ioannis Kompatsiaris, Efstratios Gavves
Abstract:
Autoregressive video diffusion models have enabled the generation of arbitrarily long videos by removing conditioning on future frames, thus greatly improving computational efficiency. Yet, they suffer from error accumulation over time, as the denoised sequence gradually drifts away from the conditioning distribution seen during training. Recent advances attempt to reduce this error by anchoring each generated frame to the learned manifold of real ones. However, even when all generated individual frames lie close to the real manifold, there are trajectories which the model lacks sufficient knowledge to continue without exiting it, thus reaching a terminal point. To prevent the model from being trapped in terminal points, we start from the hypothesis that for well‑modeled future trajectories the distribution of the predicted noise should match the one of the forward noising process. To enforce such a prior at test time, we introduce Terminal points Avoidance through Noise Guided Optimization (TANGO), which uses the diffusion model as a critic of its own outputs, by predicting one step forward and requiring an isotropic Gaussian noise prediction. We use the deviation from this expected noise distribution to search for an alternative trajectory that does not lead to a terminal point. Our approach achieves a 3.1% absolute improvement on VBench over state‑of‑the‑art, while reducing Fréchet Video Distance by 28.3% on average across 15s videos. Our code is available on https://mever‑team.github.io/tango.
Authors:Yuxuan Chen, Haipeng Xie, Shuo Dai, Ruoyi Xu, Zhaohong Bie
Abstract:
Rapidly shifting operational scenarios driven by uncertain Distributed Energy Resource (DER) profiles render conventional distribution network optimization methods either computationally expensive or poorly generalizable. This paper introduces GridRAG, a pioneering retrieval‑augmented framework that transforms optimization into a ``retrieve‑and‑refine'' paradigm. GridRAG first embeds scenario features and optimal solutions into a joint representation space to ensure semantic consistency. Based on the hybrid semantic information, the similar historical scenarios are then retrieved from a pre‑constructed database. Then an SDEdit‑style diffusion module is integrated to refine retrieved solutions by modeling the conditional distribution over near‑feasible manifolds. This process effectively pulls retrieved solutions into near‑optimal attraction basins, providing a high‑quality warm‑start for the final solver. Validated on three optimization tasks across four standard topologies, GridRAG demonstrates superior cross‑scenario generalization and a multi‑fold speedup in solution time compared to existing learning‑based and model‑based baselines. Our code is available at https://github.com/YuxuanCEE/GridRAG.
Authors:Weitao Xiong, Tianyu Liu, Peng Li, Kok Chung Chua, Toa Chean Khim, Pu Wang, Hongfei Xue
Abstract:
High‑fidelity simulation of mmWave radar signals for dynamic human motion is valuable for developing radar‑based human sensing models; yet collecting accurately labeled measurements for a specific deployment site remains expensive. We present HybridSim, a physics‑learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration, explicitly decoupling propagation into two components. To parameterize the human subject, we use a tri‑plane representation to extract human features and a Graph Convolutional Network to stabilize optimization and mitigate gradient instability. The direct signal path is modeled via an inverse‑rendering formulation with a microfacet BRDF to capture primary surface reflections. In parallel, the indirect path is approximated by combining 3D Gaussian Splatting with a virtual‑receiver geometry to fit and reproduce site‑specific multipath interference patterns, achieving substantially lower computational cost than explicit full ray tracing. Experiments in a fixed‑room setting show improved agreement with a physically based reference and consistent gains on downstream radar‑based human sensing tasks when using HybridSim for site‑specific data augmentation.
Authors:Oussama Berhili, Yassine Ouzar, Larbi Boubchir
Abstract:
We present a multimodal framework for Ambivalence/Hesitancy (A/H) recognition in video, developed for the ABAW11 challenge at ECCV 2026. The proposed approach fuses textual, acoustic, and visual modalities extracted from the BAH dataset using three pretrained encoders: F2LLM‑v2‑0.6B for transcripts (1024‑d), WavLM‑Large for audio (1024‑d), and VideoMAE V2 for facial video (768‑d). We first establish comprehensive unimodal baselines using classical classifiers (MLP, Random Forest, GBDT), each optimized via Optuna, and obtain a best unimodal Macro F1 of 0.6659 on the test set using text features alone ‑‑ substantially outperforming the zero‑shot Video‑LLaVA baseline (Macro F1: 0.2827). Building on these baselines, we propose a multimodal fusion architecture that combines bidirectional cross‑attention across all three modalities with a Gated Multimodal Unit (GMU), with both architectural and optimization hyperparameters selected through a 50‑trial Optuna search. This model achieves a Macro F1 of 0.7394 on the validation set, a relative improvement of 11.0% over the best unimodal baseline, confirming that explicit cross‑modal interaction captures complementary cues that no single modality provides in isolation. Final predictions on the official, unlabeled private test set are generated using this model and submitted according to the challenge protocol. Code is publicly available at https://github.com/yassineouzar/IUSD_AH/
Authors:Yilai Liu, Xin Zhang, Shiyuan Zhang, Hongyang Du
Abstract:
Maintaining recurring character identities across scene transitions and long temporal gaps is a central challenge in narrative long video generation. Methods targeting global consistency often retrieve memory using cues that are not aligned with character identity preservation, while recent character‑centric variants still rely on coarse frame‑level kv memory that entangles identity with incidental visual factors and lacks a continuous update mechanism under limited memory capacity. To address these limitations, we propose SlotMem, a character‑addressable internal memory framework for multi‑character narrative long video generation. Specifically, SlotMem uses a Character‑Semantic Probe to localize character‑relevant visual tokens from cross‑attention responses, and a Memory Encoder to compress DiT tokens into compact role‑wise slot memory. As generation proceeds, a Memory Writer conservatively updates each character's memory with new observations, while Character‑Wise Cross‑Attention retrieves the role memory and injects it only into localized tokens of the same character. Experiments on multiple narrative long video generation benchmarks show that SlotMem improves long‑range character consistency over existing baselines, while maintaining comparable video quality. Our code is available at https://github.com/YilaiLiu‑HKU/SlotMem.
Authors:Yujie Li, Jiancheng Pan, Zhiwei Wei, Jiuniu Wang, Mugen Peng, Wenjia Xu
Abstract:
Remote sensing offers an unparalleled vantage point for observing the Earth's long‑term surface evolution, yet it demands that a model not only perceive land cover at isolated moments, but also track changes, memorize evolution histories, and reason across time and space. However, existing studies lack a systematic evaluation that dissects these distinct competencies. To fill this gap, we introduce ChronoBench, a multidimensional benchmark that decomposes this task into four progressive cognitive levels (i.e., Land Cover Perception, Temporal Recognition, Long‑Term Memory, and Spatio‑Temporal Reasoning). The ChronoBench comprises 12 sub‑tasks and 17,689 rigorously validated QA (Question‑Answer) pairs. Extensive evaluations reveal that mainstream MLLMs fall drastically behind human experts, with Long‑Term Memory emerging as the most critical bottleneck. Motivated by this finding, we further propose GeoChrono, an MLLM with enhanced capabilities for tracing, memorizing, and reasoning about long‑term geographic evolution. Leveraging the physical prior that geographic parcels remain spatially fixed while their semantics evolve, we design a Temporal Trajectory Encoder~(TempEnc) that constructs per‑location temporal trajectories for dedicated land cover evolution modeling, and we introduce a Coarse‑to‑Fine Token Compressor~(C2FComp) that adaptively preserves dynamic regions while compressing the static background. To support training, we also construct ChronoInstruct, a 104K‑sample instruction‑tuning dataset spanning all competency levels for training. GeoChrono achieves state‑of‑the‑art performance on ChronoBench, surpassing the leading commercial MLLMs by over 20%, while C2FComp reduces visual tokens by over 56% while retaining GeoChrono's 94.6% performance. The code and data will be available at https://github.com/IntelliSensing/GeoChrono
Authors:Zhichao Yang, Tianjiao Gu, Zhixianhe Zhang, Xiangfei Sheng, Pengfei Chen, Leida Li
Abstract:
Personalized Image Aesthetic Assessment (PIAA) aims to predict aesthetic ratings of images that vary across individuals. The aesthetic preferences manifest to different extents across distinct visual stimuli and exhibit cohort‑specific patterns. Motivated by the above fact, this paper presents a Multimodal Large Language Model (MLLM)‑based approach, which models individual aesthetic preferences by Preference‑Rich sample mining and Aesthetically‑resonant Cohort merging (PRAC). Specifically, PRAC first identifies preference‑rich samples by analyzing both Collective Controversy and Personalized Deviation of images, maximizing the utility of limited user data. Based upon the preference‑rich samples, cross‑user preference similarities are measured by comparing preference embeddings. Then, a cohort‑based model merging strategy, is proposed by aggregating preference patterns from aesthetically‑resonant users, which further enhances the personalization for the target individual. Extensive experiments and comparisons on four benchmark PIAA databases demonstrate the superiority of the proposed PRAC model over the state‑of‑the‑arts. The code and model will be public at https://github.com/yzc‑ippl/PRAC.
Authors:Youngho Kim, Hoonhee Cho, Jae-Young Kang, Kuk-Jin Yoon
Abstract:
Feature tracking plays a fundamental role in understanding scene motion and supports various downstream tasks. Event cameras, with their high temporal resolution and asynchronous sensing, enable low‑latency and motion‑robust perception, making them well‑suited for feature tracking under fast and non‑linear motion. However, existing event‑based feature tracking methods rely on fixed heuristic rules based on hand‑tuning for event accumulation. Such strategies fail to adapt to diverse motion dynamics, leading to degraded performance under abrupt motion changes or low‑motion scenarios. In this paper, we model event accumulation as a sequential decision‑making problem and introduce reinforcement learning (RL) framework to adaptively control the accumulation process for online event‑based feature tracking. Our approach trains a RL agent that decides whether to continue accumulating events or to perform tracking inference based on motion cues. The proposed adaptive temporal agent enables dynamic adaptation to varying motion patterns without relying on hand‑crafted rules. Furthermore, we introduce a Dynamic Event‑based Tracking (DEFT) dataset with dynamic motion distributions to evaluate the robustness of the feature tracking. Extensive experiments demonstrate that integrating our plug‑and‑play framework to existing feature tracking methods consistently outperforms heuristic‑based approaches, improving robustness under dynamic motion while offering a better balance between tracking accuracy and efficiency. Our project codes and datasets are available at https://github.com/kmax2001/GoSTOP
Authors:Yunpeng Bai, Haoxiang Li, Qixing Huang
Abstract:
Diffusion Transformers have recently achieved strong performance in video generation, yet controlling scene geometry under viewpoint changes and camera motion remains challenging. In this work, we revisit the role of positional encoding in video diffusion transformers and show that it provides a useful spatial bias for geometry‑aware control. Specifically, if reference tokens are encoded according to their projected locations in the target view, the denoising model is encouraged to retrieve content from position aligned regions of the input video. Building on this observation, we introduce a geometry‑aware cross‑attention mechanism that enables target video latent tokens to attend to structured context tokens derived from reference images or frames. To establish correspondence between the reference content and the target camera trajectory, we equip the context tokens with a projected positional encoding scheme that combines target‑view 2D reprojection with depth‑aware disambiguation. At the same time, we preserve the original spatiotemporal positional encoding of the generated video latent, allowing geometric guidance to be injected while maintaining consistency with the video model's native latent structure. The resulting framework provides a simple and effective approach for controllable video generation. It improves spatial controllability in viewpoint‑dependent editing tasks, including camera re‑trajectory, novel‑view video synthesis, and geometry‑aware video editing, while preserving the generative prior of the underlying video diffusion model. The code is available at: https://github.com/MTLab/PE‑Field.
Authors:Lichao Mou, Shilan Zhang, Chunlei Li, Bingcong Yan, Jingliang Hu, Yilei Shi, Shengwu Xiong, Xiao Xiang Zhu, Lei Li, Yaxiong Chen
Abstract:
Large vision‑language models (LVLMs) can be adapted to specialized medical imaging tasks via parameter‑efficient fine‑tuning approaches such as low‑rank adaptation (LoRA), leading to a growing ecosystem of expert models tailored to specific imaging modalities and clinical scenarios. However, deploying multiple expert LVLMs in practice incurs substantial computational and operational overhead. Model merging provides a promising solution by consolidating multiple experts into a single model without retraining, yet it remains largely unexplored in the medical domain. In this work, we present the first systematic study of model merging for medical LVLMs. We introduce MergeMedBench, a comprehensive benchmark spanning eight imaging modalities and diverse clinical task types, comprising 16 LoRA fine‑tuned models built upon two mainstream architectures. We conduct an extensive evaluation of existing merging methods and further propose winner‑take‑all, a simple and hyperparameter‑free approach that retains only the most dominant parameters across expert models. By preserving the critical parameters that govern model behavior and discarding weaker ones, our method avoids the information dilution inherent in averaging‑ or alignment‑based strategies. Despite its simplicity, winner‑take‑all consistently outperforms existing approaches, offering both a new perspective on LoRA merging and a strong practical baseline for future research.
Authors:Wenzhe Tong, Jonathan Mi, Xili Yi, Nima Fazeli, Xiaonan Huang
Abstract:
This paper presents a scalable, open‑source visuotactile sensing system for tensegrity robots that enables six‑axis wrench estimation and contact detection. The proposed endcap sensor integrates an elastomeric shell, a 3D‑printed thermoplastic polyurethane (TPU) interface, and a rigid base housing an embedded camera and LED illumination ring. A novel gyroid‑infill bonding technique is introduced to form a durable elastomer‑TPU interface without adhesives, yielding a lightweight and modular design compatible with large‑scale tensegrity structures. A tactile‑to‑wrench neural network maps shear vector fields to six‑dimensional force and torque measurements. Experimental results demonstrate accurate and stable wrench estimation with a mean squared error (MSE) of 0.1531 on static validation data and out‑of‑domain generalization under dynamic motion. Furthermore, full‑system integration on a 12 kg tensegrity robot confirms the sensor's ability to reliably identify ground contacts. The system substantially improves the practicality of tactile feedback for tensegrity robots, offering a low‑cost, reproducible, and physically interpretable pathway toward contact‑aware proprioception and state estimation. Open source files are available at \hrefhttps://github.com/Jonathan‑Twz/tensegrity‑gelfootgithub.com/Jonathan‑Twz/tensegrity‑gelfoot
Authors:Karen Sargsyan
Abstract:
Topos causal models recast causal inference inside a topos: a causal world is a presheaf, an intervention is a characteristic map into the subobject classifier, and reasoning is carried out in the intuitionistic internal language. We give the first machine‑checked account of this 1‑topos core, in Cubical Agda, over a previously verified probability monad and do‑calculus. We build the classifier of sieves and realise the intervention \mathrmdo(X := x_0) as a characteristic map with its classification theorem; prove the sheaf gluing of independent mechanisms, which the source asserts but never proves; and machine‑check the Kripke‑Joyal forcing clauses of the internal language. In the modal layer we find and repair a gap: the three standard Lawvere‑Tierney axioms do not force a closure operator. With the missing law restored, we exhibit the double‑negation topology as a concrete instance and show that interventions and Pearl's rules are stable under every topology. Transportability of a counterfactual across a cover of regimes then coincides with this j‑stability, understood as invariance across the cover. We further add a phenomenon the programme does not consider: a machine‑checked contextuality obstruction, where pairwise‑consistent local data admit no global model. The development assumes no axioms and typechecks under Agda's ‑‑safe flag, with the ordered field discharged concretely at \mathbbQ; the scope is the presheaf (1‑topos) fragment, with type‑level sheafification and the directed lift left to future work.
Authors:Jianing Peng, Mengyu Wang, Henghui Ding, Zixiang Li, Ting Liu, Xiaochao Qu, Luoqi Liu, Yao Zhao, Yunchao Wei
Abstract:
Multi‑reference image generation aims to synthesize images by integrating attributes from multiple reference images under textual instructions. As the number of references increases, the task necessitates complex semantic comprehension, such as correctly associating attributes with the intended subjects and planing out coherent spatial arrangement between subjects and their environments. Existing approaches, which rely solely on natural language instruction, often fail to capture these complex intentions precisely, leading to semantic misalignment and inconsistent generation. We identify two key factors behind these limitations: natural language instructions are often verbose and ambiguous, and high‑quality multi‑reference data is scarce. To address these issues, we propose StructGen, which employs a structured, dictionary‑like format to encode multiple reference images, thereby enabling explicit and unambiguous specification of generation intentions. To support this design, we construct a structured dataset based on high‑quality real images and develop a corresponding training framework, along with a dedicated benchmark for challenging multi‑reference scenarios. Extensive experiments on both public benchmarks and our proposed benchmark demonstrate that StructGen consistently outperforms existing methods on both semantic alignment and detailed reference‑generation consistency, especially under complex instructions with multiple references. The code is available at https://jianingpeng0382.github.io/StructGen/
Authors:Yufeng Zhang, Zhengqi Xu, Jiajun Cui
Abstract:
This paper presents our solution to the KDD Cup 2026 Tencent UNIREC Challenge. The task requires joint modeling of multi‑domain user behavior sequences and non‑sequential multi‑field features for target‑ad pCVR prediction. We develop a Field‑Aware RankMixer (FA‑RankMixer) with dual‑stream bilinear fusion. The model first applies target‑aware DIN modules to extract user interests from multiple behavior domains. It also models recent and earlier interests separately for the longest behavior sequence. The model then forms semantic tokens based on feature fields and behavior domains and uses RankMixer blocks for cross‑token interaction. A shallow MLP stream complements the deep RankMixer stream, and a group‑wise bilinear module fuses their representations. Our final solution ranks ninth on the official leaderboard. Our code is available at https://github.com/PixelCookie‑zyf/TAAC‑2026‑SeRankMixer.
Authors:Rajat Bhattacharjya, Hyeonjong Ju, Sing-Yao Wu, Eli Bozorgzadeh, Nikil Dutt
Abstract:
Communication‑limited robots in mission‑critical scenarios such as disaster inspection and search‑and‑rescue must make reliable onboard decisions without access to remote operators or high‑capacity reasoning services. Episodic memory reuse is an attractive low‑cost fallback, but retrieval similarity does not guarantee execution validity, i.e., a retrieved action may match the current context yet be unsafe due to changed topology, insufficient battery margin, or unreliable prior outcomes. We call such high‑similarity but execution‑invalid episodes memory traps. This creates a safety‑efficiency design space where similarity only reuse minimizes fallback cost but can be unsafe, while always invoking local reasoning improves safety at high computational and energy cost. This paper presents MemoGuard, a lightweight adaptive runtime that validates episodic memories against topology, resource, and outcome contracts before reuse, invoking fallback only when validation fails. In a graph‑based corridor‑inspection simulator, MemoGuard reduces battery safety violations by 76.6% over similarity‑only top‑1 reuse while reducing fallback calls by 21.4% over always reasoning. On an NVIDIA Jetson AGX Xavier with local llama3.2:3b fallback reasoning, this corresponds to 3.67 s and 36.97 J of avoided fallback‑reasoning overhead per trial. We open‑source MemoGuard at https://github.com/hetheiin/memoguard.
Authors:Peizhen Li, Longbing Cao, Megani Rajendran, Timothy Liu, Aik Beng Ng, Simon See
Abstract:
Equipping humanoid robots with coherent and adaptable personas is crucial for fostering natural, engaging, and trustworthy human‑robot interaction (HRI). However, existing approaches often rely on static, hard‑coded identities that lack the flexibility to adapt to individual user contexts. In this paper, we present PACE (Persona Adaptation through Conversational Elicitation), a novel framework for the interactive generation and deployment of structured personas on the Ameca humanoid robot. Our system introduces an Interactive Persona Elicitation Pipeline, enabling the robot to dynamically synthesize a tailored, psychologically grounded identity through user Q&A. This elicitation process feeds into a persona prompt compilation phase, generating a structured persona prompt built upon multi‑perspective dimensions. We detail the Embodied System Integration required to translate this structured specification into expressive, multimodal humanoid behaviors. Through a comprehensive empirical HRI evaluation, we assess the impact of dynamically generated personas on user trust, perceived anthropomorphism, persona consistency, personal relevance, and interaction quality compared to a generic baseline. These contributions establish a scalable pathway for deploying personalized, interactive, and reliable identities in embodied humanoid assistants. Video demo is available at: https://lipzh5.github.io/PACE/
Authors:Hongyu Zhu, Lin Chen, Yuming Fu, Mounim A. El-Yacoubi, Mingsheng Shang
Abstract:
Electroencephalography (EEG)‑based emotion recognition captures affective neural signals with high temporal precision, but cross‑subject variability and label noise remain critical challenges to its practical healthcare deployment. Existing label‑denoising methods lack physiological grounding, while physiology‑informed approaches rely on hand‑crafted hyperparameters. To bridge these two paradigms, we propose PhyDA, a plug‑and‑play, tuning‑free framework that unifies neurophysiological priors with data‑driven label refinement. PhyDA comprises two modules. Since cross‑subject variability renders global thresholds suboptimal, the Physiological Noise Quantifier (PhyNQ) exploits a spectral slope to produce a subject‑specific noise score, providing a neurophysiologically interpretable quality assessment that naturally adapts to each individual. The Data‑Adaptive Label Refiner (DALR) directly adopts this noise score as the contamination ratio to drive a label refinement pipeline that requires no additional neural network training, thereby directly mitigating the impact of inter‑subject label noise. Extensive experiments on three public datasets (DEAP, SEED, SEED‑IV) across seven backbone architectures under strict leave‑one‑subject‑out cross‑validation demonstrate that PhyDA consistently and significantly outperforms both general and EEG‑tailored label‑denoising baselines, achieving average accuracy gains of 2.76%, 2.66%, and 3.32%, respectively. Visualization further confirms its neurophysiological interpretability and practical robustness. The source code is available at: https://github.com/HongyuZhu‑s/PhyDA.
Authors:Tong Jin, Yunpeng Liu, Shuyu Hu, Qinghua Zhang, Ruize Han, Song Wang, Feng Lu
Abstract:
Recent visual place recognition (VPR) methods based on vision transformers, particularly foundation models, have achieved remarkable recognition performance. However, these models process all visual tokens throughout the entire network, resulting in substantial computational overhead, which hinders their deployment in real‑time and resource‑constrained scenarios. A natural question thus arises: are all visual tokens necessary for VPR? To answer this question, we present the first systematic benchmark of token reduction for efficient visual place recognition. Our benchmark comprehensively evaluates representative token pruning, token merging, and hybrid pruning‑merging methods across multiple state‑of‑the‑art VPR models and diverse benchmark datasets covering urban, suburban, and natural environments. We further investigate token reduction from multiple perspectives, including recognition performance under different reduction configurations, computational complexity, inference speed, qualitative visualization, and deployment efficiency on edge devices. Through extensive experiments and in‑depth analysis, our benchmark reveals multiple important characteristics of token reduction in VPR and provides several practical insights into the trade‑offs between accuracy and inference efficiency. For example, token reduction can reduce computational cost by up to 29% and improve throughput by up to 44%, while incurring less than 1% degradation in recognition accuracy. Overall, this work establishes a comprehensive foundation for future research on token‑efficient VPR and efficient visual retrieval systems. Our codes and models will be available at https://github.com/Tong‑Jin01/TokenReduction4VPR
Authors:Damani Mguni-Coker
Abstract:
On‑the‑fly reconstruction is a key requirement for many applications in robotics and autonomous navigation. Variational Bayes Gaussian Splatting (VBGS) enables continual learning without replay buffers using Coordinate Ascent Variational Inference (CAVI), but its per‑frame iterations over all observed points make it too slow for real‑time use with strict memory and latency requirements. We present ImprovedVBGS, an accelerated framework for on‑the‑fly continual reconstruction. This is achieved primarily through (i) spatially truncated variational inference, and (ii) improved reassignment that uses forwarding, truncation and eliminates wasteful dynamic recompilation. On the NeRF synthetic dataset, we reduce mean per‑frame latency from ~84.0 s to ~0.050 s on an RTX 3070 Ti, a 1680x speed‑up while maintaining reconstruction quality.
Authors:Bibesh Pyakurel, M. G. Sarwar Murshed
Abstract:
Four‑finger SLAP fingerprints are flat live‑scan impressions of the index, middle, ring, and little fingers of one hand, used for identity verification in border control and law enforcement. No benchmark has evaluated whether multimodal large language models (MLLMs) can verify identity from SLAP images. We introduce SLAPBench, the first benchmark for MLLM‑based four‑finger SLAP fingerprint verification, built from NIST SD302b with 7,832 pairs (176 mated, 7,656 non‑mated). We evaluate four open‑source MLLMs (InternVL3‑8B, Qwen2.5‑VL‑7B, Qwen3‑VL‑8B, Gemma‑3‑12B) and the proprietary Claude Opus 4.8 under zero‑shot, task‑description, and similarity‑scoring prompts. Prompting governs verification behavior. Task‑description prompting collapses all four open‑source models to near‑100% False Accept Rate (FAR), and Gemma‑3‑12B collapses under zero‑shot as well; Claude Opus 4.8 alone resists collapse under both binary prompts, giving the best binary result (FAR = 20.2%). Similarity scoring removes collapse across the open‑source models and exposes wide capability gaps: Claude reaches AUC = 0.953 and Gemma‑3‑12B 0.837, while InternVL3‑8B is inverted (AUC = 0.590) and Qwen2.5‑VL‑7B near random (0.567). Qwen3‑VL‑8B attains perfect separation (AUC = 1.000), which we treat as a diagnostic rather than as capability: SD302b holds one SLAP capture per finger position, so mated pairs are cross‑resolution. A matched‑resolution control leaves the perfect score intact, ruling out the resolution shortcut; what cannot be excluded within SD302b is near‑duplicate detection, since a mated pair is one capture rendered twice. A fairness probe over gender, race, and age suggests disparity grows as discrimination weakens. SLAPBench establishes the first SLAP‑specific MLLM baseline and shows that prompting governs collapse while model capability governs discrimination.
Authors:Tianyi Gao, Jiayu Lin, Danielle Beaulieu, Nathan Jacobs
Abstract:
Cross‑view geo‑localization matches ground‑level observations against geo‑tagged satellite imagery. Recent methods show that sequential queries such as video clips yield richer spatiotemporal cues than single images, yet they overlook a complementary sequential modality: route descriptions ‑‑ which capture the same trajectory at a higher level of abstraction and are often the only input available (e.g., a user directing an autonomous vehicle to a pickup point). To bridge this gap, we introduce SeqGeo‑VL, a dataset of ~39K video‑text‑satellite triplets, and TrajLoc, a unified framework capable of processing both video clips and route descriptions. By leveraging both dense visual and abstract linguistic semantics, TrajLoc enables these modalities to mutually reinforce cross‑view matching. We further propose TrajMod, a lightweight module that conditions query embeddings on trajectory geometry, yielding spatially‑aware representations. Experiments show that TrajLoc achieves substantial gains over state‑of‑the‑art methods on both video and text geo‑localization. The project page is available at https://humblegamer.github.io/trajloc/.
Authors:Josef Lindl, Mariana Chaves, Damien Garreau
Abstract:
The increasing complexity of state‑of‑the‑art machine learning models has made their behavior progressively harder to interpret, spurring rapid advancements in the field of eXplainable Artificial Intelligence (XAI). Among many methods proposed, perturbation‑based approaches play a major role. By systematically altering (perturbing) input features, these approaches measure the impact on the model's predictions. For image data, traditional perturbation techniques, often involve replacing pixel values e.g., with a pre‑defined color. However, such approaches, but also more refined deterministic techniques, generate unrealistic out‑of‑distribution samples and often leave visible artifacts, which can mislead the model and compromise explanation quality. In this work, we adjust LIME, a widely used perturbation‑based method, to demonstrate how generative inpainting can improve perturbation‑based explanations for images. We achieve photorealistic perturbed samples that align better with the original data distribution and enhance explanation quality.
Authors:Zhuo Ye, Feng Zhang, Maxim Moraru, Weiyi Xia, Ying Wai Li, Yongxin Yao, Cai-Zhuang Wang
Abstract:
Exa‑PD is a highly parallelizable workflow designed for the construction of multi‑element phase diagrams (PDs). It uses standard sampling techniques, molecular dynamics (MD) and Monte Carlo (MC) as implemented in the LAMMPS package, to simultaneously sample multiple phases over a fine temperature‑composition mesh for free‑energy calculations. Parsl serves as the global workflow engine, coordinating large ensembles of MD and MC tasks to achieve massive parallelization with strong scalability. The resulting free energies of liquid and solid phases are then fed to CALPHAD modeling via the PyCalphad package to construct multi‑element PDs.
Authors:Hussein Fellahi
Abstract:
Optimization models are built in a variety of modeling languages and solved by a variety of solvers, but once a solution exists, the information needed to understand it is fragmented: each solver exposes a partial, differently named set of native diagnostics, and the modeling language has already canonicalized the formulation the user wrote. We present pyoptexplain, a practitioner‑first Python library for post‑optimality analysis of optimization models that sits above this layer. It adapts a model authored in any of five modeling front ends, namely cvxpy, Pyomo, gurobipy, docplex, and OR‑Tools, into a normalized internal representation, solves it through a choice of backends, and answers the why and what‑if questions of an optimization decision through one uniform interface. The design rests on two observations. First, a post‑optimality quantity requested from different backends for the same problem can come back as an exception, a structurally meaningless zero or a basis‑dependent value that disagrees across solvers, so reporting whatever one solver returns is unreliable. pyoptexplain reports a quantity only when both the representation and the chosen backend can justify it, and does not approximate unavailable information. Second, repeated scenario analysis can amortize its cost by extracting the model once and reusing a warm solver session across a batch of scenarios. pyoptexplain builds a single scalable what‑if interface, uniform across its modeling languages and backends and returning a certified report for every scenario, at a cost within a small constant factor of the bare solver. A reproducible computational study substantiates both claims. Source code is available at https://github.com/h‑fellahi/pyoptexplain and installation can be done through the Python Package Index https://pypi.org/project/pyoptexplain/.
Authors:Yuhao Zhou, Sheeraz Athar, Zhixian Hu, Binghao Huang, Yunzhu Li, Juan Wachs, Yu She
Abstract:
This paper presents a tactile‑reactive gripper that integrates a Visuo‑Tactile Active Palm (VTAP) and compliant, reconfigurable fingers equipped with tactile array sensors. The design exploits structured finger‑palm synergy and multi‑modal perception to achieve both robust grasping and fine manipulation. The actuated bi‑modal palm seamlessly combines long‑range visual localization with contact‑rich tactile feedback, substantially extending the system's manipulation capability. To bridge the embodiment gap between human hand motion and the heterogeneous three‑finger structure, we further propose a staged, gesture‑conditioned retargeting framework for dexterous teleoperation. Extensive experiments validate the system across a range of challenging tasks: reactive grasping of YCB and fragile objects, in‑hand syringe reorientation and plunger actuation, singulation of clustered objects down to 3 mm in diameter, and vision‑tactile peg‑in‑hole insertion. Results demonstrate that high manipulation performance can be achieved through coordinated finger‑palm interaction and multi‑modal sensing, without resorting to high degrees of freedom anthropomorphic designs. The VTAP gripper and its retargeting framework offer a practical reference architecture for dexterous gripper design, manipulation, and contact‑rich data collection in support of learning‑based approaches. Project webpage: https://yuhochau.github.io/vtap/.
Authors:Bhaskar Krishnamachari
Abstract:
Interleaving mitigates burst errors but introduces decoding delay and removes temporal error structure that a channel‑aware decoder could exploit. We consider packet‑level selection between a random linear code and the same code used with cross‑codeword interleaving, over a channel with an unknown number of on/off interferers. The receiver uses Guessing Random Additive Noise Decoding (GRAND) with a replaceable noise model and feeds aggregate channel statistics back to a Bayesian estimator at the transmitter. Once the interference amplitudes and timing parameters are estimated, the receiver's noise model is replaced: it computes hidden‑Markov‑model posterior bit‑flip probabilities and uses them to order GRAND queries. A discounted Thompson sampler selects between the two transmission modes using a goodput‑minus‑latency reward whose distribution is endogenously nonstationary: receiver adaptation, rather than channel change, alters the value of each mode. Across five simulation seeds, the interleaved mode is preferred before channel estimation converges. After the learned decoder is activated, the non‑interleaved mode becomes preferable because it achieves lower block error rate without interleaving delay. In the reference configuration, the learned noise model reduces block error rate by approximately one order of magnitude relative to ORBGRAND. Using partial channel estimates before full convergence reduces pre‑convergence block error rate by up to 4.5×. Adding model‑predicted utilities as confidence‑weighted pseudo‑observations reduces post‑transition selection of the inferior arm by approximately 65%. Under an idealized airtime conversion at a 100~MHz 5G~NR‑like symbol rate, the learning transient corresponds to a few milliseconds of occupied symbol time.
Authors:Tipu Sultan, Param Sangani, Kody Cool, Pascal Sikorski, Guangping Liu, Hadi Akbarpour, Madi Babaiasl
Abstract:
We present NeuroCommitSSM, a decision‑centric framework that models when to execute, not just what to do, for safe commit‑to‑execute control in assistive robotic manipulation. NeuroCommitSSM predicts a continuous commit‑readiness score c_t in [0,1] from synchronized electroencephalography (EEG), electromyography (EMG), and eye‑tracking (ET), and converts it into discrete commit events through dwell and hysteresis filtering. A three‑state finite‑state supervisor, HOLD‑ASSIST‑COMMIT (HAC), gates execution by requiring both a sustained commit‑readiness signal from the neural model and real‑time perception and robot‑state feasibility, including target visibility, inverse kinematics solvability, and collision‑free planning, before initiating motion. We evaluate the framework on N=32 subjects performing five activities of daily living (ADL) tasks aligned with the International Classification of Functioning, Disability and Health (ICF), using leave‑one‑subject‑out (LOSO) cross‑validation and seven sensor‑dropout scenarios (S0‑S6). NeuroCommitSSM achieves 0.950 action‑balanced accuracy with 0.75 false commit events per 1000 REST windows (FP/1k REST), and maintains low false commits and stable state transitions under sensor loss. For example, in the EEG‑only condition, it achieves 0.785 balanced accuracy and 0.29 FP/1k REST, whereas the Temporal Convolutional Network baseline produces 99.95 FP/1k REST under the same condition. Hardware‑in‑the‑loop (HIL) validation on a Kinova Gen3 arm shows that feasibility‑checked execution reduces false starts and decision instability without sacrificing task success. Supplementary materials, including code, datasets, videos, and additional analyses, are available at https://madibabaiasl.github.io/NeuroCommitSSM/.
Authors:Chanyoung Ahn, Jaesung Lee, Donhyun Hwang
Abstract:
Multisensory integration, particularly through visual and tactile feedback, plays a crucial role in enhancing audience engagement with artworks. Although recent research has increasingly explored tactile experiences in art, existing systems often lack real‑time variable stiffness modulation and depend on bulky mechanical infrastructures. In this work, we propose a novel tangible display based on a magnetic jamming mechanism, enabling real‑time, low‑noise, and low‑voltage stiffness modulation integrated into traditional sculptural artworks. Our system combines visual motion and dynamic tactile feedback within a compact standalone module, allowing audiences to interactively experience variations in the rigidity and form of features such as those found in the traditional Korean mask Hahoetal. This approach offers a new paradigm for interactive art, enabling more immersive, multisensory engagement through the fusion of cultural artifacts and modern technology. Our project page is available at https://cold‑young.github.io/jamming_tangible/.
Authors:actAVA AI, :, Haolin Chen, Leon Qi, Steve Brown, Deon Metelski, Tao Xia, Joonyul Lee, Qixuan Wang, Kevin Riley, Frank Wang, Weiran Yao
Abstract:
Healthcare spans high‑stakes communication, expert reasoning, and workflow execution, yet specialized LLMs that cover these use cases together remain limited. A healthcare model must handle patient consultation, clinical reasoning over text and images, interactive diagnosis, and electronic health record (EHR) tool use. These capabilities fail in different ways, and a narrow update for one task can degrade another. We present Cura 1T, a healthcare‑specialized LLM trained through a human‑gated self‑evolution loop. In each evolution round, a training agent plans a target capability, trains the model, evaluates benchmark trajectories, and refines the data mixture from observed failures. This data‑centered loop improves the model through targeted synthetic and curated examples rather than a single generic medical‑data update. Across the healthcare evaluation suite, Cura 1T ranks at or near the top among frontier baselines, while remaining competitive on out‑of‑domain reasoning and agentic benchmarks.
Authors:Kaihui Cheng, Zhiqiang Cai, Peng Tu, Yisong Yao, Limei Han, Libo Wu, Siyu Zhu, Tzuhsiung Yang, Yuan Qi
Abstract:
Proteins function through coordinated motion across multiple spatial and temporal scales, underpinning processes such as ligand binding, allostery, and catalysis. However, accessing long‑timescale conformational change through molecular dynamics (MD) simulations remains prohibitively expensive for systematic exploration across diverse systems. Here, we present DyneTrion, a generative protein dynamics emulator that jointly enforces geometric symmetry, structural consistency and temporal coherence within a single framework. DyneTrion uses a tri‑attention architecture that integrates invariant point attention (IPA) for SE(3)‑robust geometric updates, spatial attention anchored to a reference conformation to preserve structural integrity, and temporal attention to model correlated evolution across time frames. Across 100‑ns MD trajectory simulation benchmarks, DyneTrion reproduces MD‑derived flexibility, ensemble distributions and interaction observables while maintaining stereochemical validity during extrapolation. To evaluate long time‑scale generalization, we introduce dynamicPDB, a dataset of over 10,000 proteins with up to 1‑μs all‑atom trajectories at 10‑ps resolution and accompanying physical annotations. On microsecond trajectories, DyneTrion preserves free‑energy landscapes and metastable‑state populations, and it supports large conformational propagation in apo‑to‑holo transitions and fast folders. Together, DyneTrion provides a scalable path from static structure prediction toward time‑resolved, ensemble‑faithful protein modeling. The code is publicly available at https://github.com/fudan‑generative‑vision/DyneTrion
Authors:Zhiyuan Zhao, Bin Wang, Linke Ouyang, Yiqi Lin, Pan Zhang, Xiaoyi Dong, Jiaqi Wang, Conghui He
Abstract:
In this paper, we propose MLLM‑DataEngine, a novel closed‑loop system that bridges data generation, model training, and evaluation. Within each loop iteration, the MLLM‑DataEngine first analyzes the weakness of the model based on the evaluation results, then generates a proper incremental dataset for the next training iteration, and enhances the model capability iteratively. Compared with previous instruction fine‑tuning dataset collection methods which are separate from the benchmarking, MLLM‑DataEngine shows better targeting and can improve MLLMs's capabilities more effectively. Firstly, we propose an Adaptive Bad‑case Sampling module, which can effectively analyze model weakness based on the benchmarking results and adjust the generation of incremental datasets flexibly. Secondly, in order to ensure high‑quality data for specific capability types, the most representative in‑context examples and abundant information are provided to GPT‑4, which helps GPT‑4 fully comprehend the model's weakness and further guarantees high‑quality generated data. Through extensive experiments, we find MLLM‑DataEngine could boost the MLLMs capability in a targeted and automatic manner without human participants. We hope MLLM‑DataEngine could be a general solution for the following MLLMs data curation. Code, data, and model are available at https://github.com/opendatalab/MLLM‑DataEngine.
Authors:Zezhong Qian, Xiaowei Chi, Chak-Wing Mak, Tianze Zhou, Ruibin Yuan, Yuhan Rui, Hengzhe Sun, Zhuoqun Wu, Yuming Li, Siyuan Qian, Sirui Han, Shanghang Zhang
Abstract:
Video models are evolving into vision foundation models, yet they still lack human‑like multi‑step reasoning. Streaming autoregressive diffusion models are efficient but limited in reasoning, while bidirectional diffusion enables global revision with high inference costs due to dense frame‑level denoising. Both paradigms struggle to achieve logical consistency and low‑latency streaming for complex reasoning tasks. We propose HDR (Hierarchical Denoising for Visual Reasoning), a unified framework that integrates hierarchical latents into causal video generation for multi‑step reasoning. HDR organizes video latents into a tree‑structured hierarchy, enabling coarse‑to‑fine reasoning before streaming output. Coarse denoising layers preserve uncertain hypotheses for global planning, while finer layers progressively refine them into concrete visual states. A sparse hierarchical attention pattern (SHAP) further reduces temporal attention costs. We introduce a level‑stratified multi‑step video reasoning benchmark with out‑of‑distribution cases, covering six tasks: maze navigation, Tower of Hanoi, one‑line drawing, sliding puzzle, Sokoban, and water pouring. Compared with streaming autoregressive diffusion baselines, HDR improves success from 34.22 to 60.29 (76.2% relative gain) and increases average progress from 76.00 to 89.56, demonstrating more consistent reasoning trajectories. HDR maintains low‑latency streaming at 0.70 seconds per latent, achieving 54.2 times faster inference than bidirectional diffusion. It also retains 82.9% of full‑data performance with only 2% training data, compared with 52.0% for bidirectional diffusion. Real‑world robot experiments further demonstrate HDR's potential for physical interaction and world modeling. Project demo: https://hierarchical‑diffusion‑reasoning.github.io/.
Authors:Yushi Huang, Xiangxin Zhou, Jun Zhang, Liefeng Bo, Tianyu Pang
Abstract:
MeanFlow generators achieve fast few‑step sampling by predicting average velocities over time intervals, making them attractive for efficient generation. Reinforcement learning (RL) has become a powerful way to align diffusion and flow models with human preferences and task‑specific objectives. In particular, DiffusionNFT offers an efficient forward‑process RL framework that does not require reverse‑process trajectories or likelihood estimation. However, applying such RL methods to MeanFlow remains underexplored. DiffusionNFT optimizes instantaneous velocities, whereas MeanFlow samples with average velocities. To bridge this gap, we introduce MeanFlowNFT. Inspired by the MeanFlow identity, which bridges average and instantaneous velocities, we construct an induced instantaneous‑velocity predictor. We apply the DiffusionNFT objective to this predictor, making reward optimization well‑defined for MeanFlow. Sampling remains based on the average velocity, preserving MeanFlow's fast few‑step generation. We further prove that MeanFlowNFT inherits DiffusionNFT's strict policy‑improvement guarantee. Experiments on image and video generation show that MeanFlowNFT consistently improves baselines. Moreover, it outperforms prior state‑of‑the‑art RL‑tuned few‑step generators on most metrics (6 of 8 on SD3.5‑M), and can even surpass multi‑step RL‑tuned diffusion while using only a few sampling steps. For instance, on Wan 2.1, 4‑step MeanFlowNFT reaches a VBench score of 84.33, surpassing 50‑step LongCat‑Video RL (82.57).
Authors:Yuyao Zhang, Junjie Gao, Zhengxian Wu, Jiaming Fan, Jin Zhang, Shihan Ma, Yao Yao, Weiran Qi, Chuyan Jin, Guiyu Ma, Xingzhong Xu, Kai Yang, Ji-Rong Wen, Zhicheng Dou
Abstract:
Recent advances in Tool‑Integrated Large Language Models have made web search a core capability of information‑seeking agents. However, as interaction histories grow, agents increasingly struggle to track task progress. When search attempts fail to yield useful evidence, current single‑ and multi‑agent systems can become trapped in repetitive loops, wasting search budgets and ultimately compromising the quality and completeness of the final output. We introduce SearchOS, a system‑level multi‑agent framework that turns fragile, implicit search progress into explicit, persistent, and shared state. First, we formulate open‑domain information seeking as relational schema completion with grounded citations, where agents discover entities, populate attributes across linked tables, and anchor each value to source evidence. Then we design Search‑Oriented Context Management (SOCM), which externalizes the evolving state into Frontier Task, an Evidence Graph, a Coverage Map, and Failure Memory. Built on SOCM, SearchOS applies a pipeline‑parallel scheduling mechanism that overlaps the execution of sub‑agents and continuously refills freed slots with tasks targeting unresolved coverage gaps to improve utilization and throughput. To schedule and control the execution of search agents, SearchOS introduces a Search Tool Middleware Harness that intercepts model and tool interactions to record grounded evidence and react to stalls or budget exhaustion, and provides a reusable hierarchical skill system comprising strategy and access skills to augment the agents' search process and avoid repeating failed search patterns across runs. On WideSearch and GISA, SearchOS leads all metrics among the evaluated single‑ and multi‑agent baselines, paving the way toward robust information‑seeking collaboration.
Authors:Maya Varma, Jean-Benoit Delbrouck, Sophie Ostmeier, Akshay Chaudhari, Curtis Langlotz
Abstract:
Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image‑text pairs. Our work focuses on a class of captioning errors that we refer to as systematic misalignments, where a recurring error in MLLM‑generated captions is closely associated with the presence of a specific visual feature in the paired image. Given a vision‑language dataset with MLLM‑generated captions, our aim in this work is to detect such errors, a task we refer to as systematic misalignment detection. As our first key contribution, we present Symbal, which utilizes a structured, dual‑stage setup with off‑the‑shelf foundation models to identify systematic misalignments and summarize results in natural language. As our second key contribution, we introduce SymbalBench, a benchmark designed to evaluate automated methods on our proposed task. SymbalBench consists of 1.7 million image‑text pairs from two domains (natural and medical images), organized into 420 vision‑language datasets with annotated systematic misalignments. Symbal exhibits strong performance on this benchmark, correctly identifying systematic misalignments in 63.8% of datasets, a nearly 4x improvement over the closest baseline. We supplement our evaluations on SymbalBench with real‑world evaluations, showing that (1) Symbal can accurately surface systematic misalignments in captions generated by four MLLMs and (2) Symbal is a powerful tool for auditing off‑the‑shelf image‑caption datasets. Ultimately, our novel task, method, and benchmark can aid users with auditing MLLM‑generated captions and identifying critical errors, without requiring access to the underlying MLLM. Code is available at https://github.com/Stanford‑AIMI/Symbal.
Authors:Patrick Phuoc Do, Chau M. Ta, Chaoli Wang
Abstract:
Multimodal large language models (MLLMs) are increasingly used to interpret visualizations, yet current evaluations remain largely chart‑centric and provide limited evidence of understanding of scientific visualization (SciVis). We benchmark six MLLMs on the scientific visualization literacy assessment test, a standardized SciVis literacy assessment comprising 49 items based on 18 scientific visualizations and illustrations, spanning 8 techniques and 11 task types. We evaluate three closed‑source and three open‑source models under a closed‑world protocol and compare their performance using data from 485 human participants. Results show that current MLLMs do not exhibit uniform SciVis literacy. Gemini is the strongest model overall, exceeding the human mean across the evaluated subsets, whereas the open‑source models remain below the human baseline. Performance is highly uneven across techniques and tasks: models perform best on scientific illustration, search, and spatial understanding, but struggle on texture‑based and integration‑based visualizations and on quantitative estimation. Error analysis reveals recurring failures in fine‑grained quantitative estimation, flow‑direction interpretation, and grounded encoding interpretation. These findings position SciVis literacy as a necessary benchmark dimension for evaluating multimodal AI systems. Our code and model outputs are publicly available at https://github.com/patdmp/mllm‑scivis‑lit‑benchmark.
Authors:Byeongho Heo, Jaehui Hwang, Sangdoo Yun, Dongyoon Han
Abstract:
On‑policy distillation is an alternative post‑training method in reinforcement learning that alleviates the constraints imposed by reward models by providing token‑level supervision from a teacher model. Although on‑policy distillation has been studied and applied across various settings, its fundamental design remains underexplored. In this paper, we introduce a new distillation reward, termed the delta signal, instead of directly imitating the teacher's output distribution. The delta signal is defined as the difference between the teacher model and its base model prior to instruction tuning for reasoning capability. It therefore captures the changes induced by reasoning tuning and provides a more direct signal for transferring reasoning capabilities. Using extensive empirical evidence, we show that the delta signal substantially improves on‑policy distillation and refer to the new distillation method as On‑Policy Delta Distillation (OPD^2). Experiments across mathematics, science, and code‑reasoning benchmarks demonstrate that OPD^2 consistently outperforms conventional on‑policy distillation, enabling reasoning LLMs to achieve strong performance with only a short post‑training period. Code will be available at https://github.com/naver‑ai/opd2
Authors:Shen Zhou, Jinghui Zhang, Wenbo Huang, Xuwei Qian, Zhen Wu, Guangwen Peng, Zhiyuan Li, Ding Ding, Dian Shen, Fang Dong
Abstract:
All‑in‑one image restoration aims to recover clean images degraded by multiple corruption types using a single unified model. Existing methods typically rely on image‑level prompts or shared guidance to handle diverse degradations. However, such a paradigm becomes inadequate when degradations are spatially heterogeneous or even coexist in mixed forms within a single image. Yet spatially adaptive guidance alone is not sufficient, since accurate restoration also requires each spatial query to reliably aggregate complementary information from local neighborhoods and global contexts. To this end, we propose QuReC, a unified framework for all‑in‑one image restoration. QuReC consists of a Degradation‑Guided Query Reconstruction Module (DQRM) and a Local‑Global Response Calibration Module (LGRCM). Specifically, DQRM matches each spatial query against a degradation prototype space to reconstruct a query‑specific degradation‑aware representation, thereby providing fine‑grained spatially adaptive restoration guidance. To further stabilize this query‑wise matching process, we introduce a weakly supervised prototype matching learning strategy to improve optimization stability and degradation semantic consistency. Meanwhile, LGRCM performs local‑global dual‑branch aggregation and calibrates the aggregated responses with learnable priors, improving the reliability of feature aggregation and the coordination between local detail modeling and global context modeling. Extensive experiments demonstrate that QuReC achieves superior performance on multiple all‑in‑one image restoration benchmarks. The code is released at https://github.com/zhoushen1/QuReC.
Authors:Yao Cheng Li, Ana Larrañaga, Steven L. Brunton, Urban Fasel
Abstract:
Many engineering problems involve phenomena whose governing equations are poorly characterized or only partially known. Surrogate modeling techniques such as neural networks can capture the behavior of these systems, but they typically demand large training datasets that are difficult to obtain in engineering contexts and yield models with limited physical interpretability. The Sparse Identification of Nonlinear Dynamics (SINDy) method addresses both limitations by performing sparse regression over libraries of candidate nonlinear terms, recovering interpretable governing equations from comparatively small datasets. Although SINDy has been demonstrated extensively on canonical benchmark systems, its application to practical engineering problems is less widely documented. This tutorial introduces the SINDy method and progressively builds toward its main extensions, from noise‑robust weak‑form and ensembling‑based variants to constrained and parametrizable formulations. The paper and the accompanying tutorial (available at https://github.com/paullililili/SINDy4Engineers) is organized in three parts: the first introduces the standard SINDy algorithm and progressively extends it, inviting readers without prior knowledge to follow each step and adapt the methods to their own problems; the remaining two parts present detailed case studies on (1) the system identification of an unmanned aerial vehicle and (2) a chaotic thermosyphon heat exchanger. Through these examples, we aim to demonstrate that SINDy is simple to implement yet flexible enough to serve as a valuable identification tool for advanced engineering applications.
Authors:Susie Lu, Haonan Chen, Weirui Ye, Yilun Du
Abstract:
Predictive world models enable robots to plan by imagining the outcomes of their actions, but their value for control hinges on generating many rollouts quickly. This creates a bottleneck for diffusion‑based world models: multistep sampling makes each rollout expensive, limiting large‑scale action search at inference time. We introduce DriftWorld, an action‑conditioned world model based on drifting generative models. Rather than denoising iteratively at inference, DriftWorld learns an action‑conditioned drift during training, allowing it to generate future frames from the current observation and a candidate action sequence in a single forward pass at 30+ fps, which is 17x faster on average than diffusion based baselines. We evaluate DriftWorld on standard vision‑based robotic manipulation benchmarks, including Bridge‑V2, RT‑1, Language Table, Push‑T, and Robomimic. By producing rollouts that are both accurate and fast, DriftWorld achieves state‑of‑the‑art decision‑making performance with far less inference time than diffusion‑based world model baselines. Beyond online control, DriftWorld can also serve as an offline simulator for ranking real‑world robot policies, with rollout‑based scores correlating with ground truth at up to 0.99. These results show that drifting models are a strong fit for robot world modeling, where fast, high‑quality imagination directly supports planning and policy evaluation.
Authors:Saad Ejaz, Miguel Fernandez-Cortizas, Javier Civera, Holger Voos, Jose Luis Sanchez-Lopez
Abstract:
CAD‑to‑image alignment aims to estimate an object's 9D pose (rotation, translation, and anisotropic scale) from a single RGB image, enabling applications in robotics and augmented reality. Recent zero‑shot methods use visual foundation models to match image regions to CAD models, yet typically their correspondences are appearance‑driven and degrade under occlusion or sim‑to‑real domain shift. To address these limitations, we introduce SUFLECA (Scaling Up Feature LEarning for CAD Alignment), a weakly‑supervised framework for zero‑shot CAD alignment with two key contributions. First, SUFLECA scales up geometry‑grounded feature learning from pretrained visual representations through Normalized Object Coordinates (NOCs) supervision on 674K images spanning 12 real and synthetic datasets, learning compact geometry‑aware features that generalize across domains. Second, we propose a geometrically consistent matching algorithm that establishes reliable one‑to‑one CAD‑to‑image correspondences. Together, these contributions enable accurate, sub‑second alignment per object instance without iterative pose refinement. On ScanNet25k, SUFLECA achieves 33.4%/42.3% category/instance accuracy, outperforming, with a smaller computational footprint, the strongest zero‑shot baseline by 10.3/12.2 percentage points and, for the first time on this benchmark, even surpassing fully supervised methods. Code is available at: https://github.com/snt‑arg/SUFLECA
Authors:Xiao Lin, Xiaohu Huang, Kai Han
Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated substantial promise in spatial understanding. Existing works typically incorporate prior knowledge extracted from a pre‑trained foundation model to further enhance the spatial awareness of MLLMs. In this paper, we first reveal that when integrating diverse foundation models into MLLMs, different models provide complementary spatial priors that benefit different tasks. Motivated by this, we propose ViPS, a novel multi‑model prior framework designed to fully unleash the potential of incorporating multiple Visual Priors from diverse models into MLLMs for Spatial understanding. Specifically, ViPS introduces an Efficient Prior Proxy to generate multiple foundational priors with minimal inference overhead, and a Dynamic Prior Fusion mechanism to achieve harmonious and context‑aware prior fusion and injection from the prior proxies. Extensive experiments demonstrate that ViPS successfully harmonizes diverse visual priors, establishing new state‑of‑the‑art performance across multiple complex spatial reasoning and 3D spatial understanding benchmarks. Project page: https://visual‑ai.github.io/vips
Authors:Wenqi Si, Gongyang Li, Shixiang Shi, Weisi Lin
Abstract:
Weakly‑supervised RGB‑D Salient Object Detection (SOD) is explored to reduce the heavy burden of pixel‑level annotations. But scribble annotations lack the structure and details of objects, resulting in inaccurate saliency maps. In this paper, we propose a novel scribble‑supervised RGB‑D SOD method, consisting of a Segment Anything Model (SAM)‑driven pseudo annotation generation method (\emphSAM‑PAG) and a state space interaction‑based conditional diffusion model (\emphS^2Diff). Specifically, SAM‑PAG is tailored to address the issue of sparse supervision information. In SAM‑PAG, we adopt the advanced SAM to expand sparse scribbles to dense pixel‑level pseudo annotations through the dual‑branch structure and the consistency of segmentation masks. In S^2Diff, we adopt the diffusion model to iteratively refine the noisy saliency maps with the guidance of conditional information, generating accurate saliency maps. Naturally, the core of our S^2Diff lies in the acquisition of conditional features and the denoising of saliency maps. For the former, we employ a cross‑modal conditional generation module to interweave cross‑modal features through frequency integration and implicit‑explicit state space interaction, effectively achieving global conditional features. For the latter, we employ a context injection module to mitigate noise interference and to enhance object information with the conditional context. With the close cooperation of SAM‑PAG and S^2Diff, our method outperforms relevant scribble‑supervised methods and achieves competitive performance compared to fully‑supervised methods on seven datasets. The code and results of our method are available at https://github.com/Switch457/WeakS2Diff_SOD.
Authors:Yuchen Ren, Zhengyu Zhao, Chenhao Lin, Bo Yang, Chao Shen
Abstract:
Vision‑Language Pre‑training Models (VLPMs) are known to be vulnerable to adversarial attacks. Recent transferable attacks on VLPMs have followed a common pipeline with complicated loss functions or multi‑stage text/image attacks. However, in this paper, we demonstrate that such a sophisticated attack pipeline can be simpler yet more successful. Specifically, we identify three previously overlooked issues caused by inappropriate cross‑modal interactions and excessive operations. To address them, we propose the Simple Vision‑Language Attack (SimVLA) pipeline, which observably improves transferability and efficiency. Experiments on four datasets and three downstream tasks validate the superiority of our pipeline. For instance, on Flickr30k text‑image retrieval dataset, our SimVLA outperforms the SOTA baseline in R@1 transferability by 8.01%‑14.71%, while consuming only about 35.73% of the time and 46.26% of the max VRAM. Overall, the superiority of our SimVLA highlights the importance of leveraging domain knowledge (e.g., our proposed cross‑modal word identification), while blindly pursuing intricate operations (e.g, complex loss functions and redundant multi‑stage designs) may even be harmful. We hope our SimVLA can serve as a simple yet effective backbone for future extensions. Code is available at https://github.com/RYC‑98/SimVLA.
Authors:Basim Azam, Hossein Rahmani, Naveed Akhtar
Abstract:
State‑of‑the‑art flow based text‑to‑image (T2I) models exhibit remarkable generative abilities but remain vulnerable to producing unsafe content. Prior safety efforts range from concept erasure and prompt filtering to classifier‑based gating. However, simple techniques like parameter efficient adaptations of the models easily bypass such guardrails. We introduce a unique principled approach that achieves safety by regulating the model's attention dynamics through inference‑time introspection, exhibiting intrinsic robustness. Our method analyzes and rebalances attention activations throughout image synthesis, steering generations away from unsafe concepts while preserving semantic alignment. This introspective control ensures safety of deployed models. Across standard and adversarial safety benchmarks, our approach achieves remarkable safety scores while maintaining or even improving alignment and perceptual quality. Our results reveal that attention‑space regulation offers a considerably more promising path to safer diffusion transformer based image generation than the existing concept erasing mechanism.Our code can be accessed at https://basim‑azam.github.io/iam/
Authors:Siwoo Lim, Sunjae Yoon, Gwanhyeong Koo, Hyeonseo Yun, Chang D. Yoo
Abstract:
While recent flow‑matching 3D generative models (e.g., VecSet) adopt structured representations, their tokens share global context, causing conventional training‑free editing to suffer from semantic artifacts such as collapsed preserved regions or incomplete transformations. To address this, we propose TanGO, a training‑free framework that enables adaptive per‑token steering in the tangent space of generative dynamics. To realize this selective control, we formulate a one‑step optimal control rule and determine the strength of each token's control signal using a von Mises‑Fisher inspired directional discrepancy derived from the source and target velocity fields. Experiments show that TanGO substantially reduces structural artifacts and achieves state‑of‑the‑art performance, outperforming existing 3D editing baselines. The code is publicly available at https://github.com/siw00‑lim/TanGO.
Authors:Sizhong Qin, Yi Gu, Yao Jiang, Ao Cai, Changjian Zhou, Shaoxuan Shuai, Jiachang Wang, Tianhao Shen, Yueqiang Li, Xinhao Li, Li Zeng, Yueshi Chen, Dachen Gao, Genrong Xu, Wenjie Liao, Xinzheng Lu
Abstract:
Addressing a structural‑engineering request requires more than a single answer; it requires a chain of interdependent artifacts: interpreted requirements, a computable model, validation records, solver outputs, code‑check records, and a final report. Evaluations centered on question answering or script generation rarely verify this complete evidence chain and may therefore reward fluent outputs even when the underlying engineering workflow is incomplete, internally inconsistent, or non‑executable. To address this limitation, we present StructureClaw, an artifact‑centered workbench in which LLM agents operate through governed engineering skills, typed tools, shared artifact state, and local analysis backends. We also introduce StructureClaw‑Bench, an executable benchmark of 150 controlled scenarios spanning standard workflow execution, interactive robustness, and multimodal structural‑model reconstruction. A scenario succeeds only when all required artifact‑ and execution‑level assertions pass in a single run. Across ten agent‑model configurations, each evaluated on the same 50 standard cases, the average Success Rate rises from 56.8% with the generic‑skill baseline to 88.6% with the full automatic workflow. The interactive and multimodal evaluations identify two prominent remaining challenges: safe handling of invalid numerical inputs and fixture‑consistent reconstruction of structural models. These findings show that artifact‑centered evaluation can expose workflow‑level failures that are difficult to identify from final responses alone, providing a more rigorous basis for evaluating and improving structural‑engineering agents. The code and benchmark are available at https://github.com/structureclaw/structureclaw.
Authors:Lucas Bergholdt Hansen, Federico Torrielli, Filippo Tonini, Lukas Galke Poech
Abstract:
LLM‑based agents are increasingly deployed in multi‑agent environments whose incentives can shape their behavior. We introduce The Energy Society, a minimal survival economy for studying how competitive and cooperative incentives affect emergent behavior when inference cost is directly tied to survival: Agents spend energy based on model size when generating tokens, regain energy by completing jobs or receiving donations, and deactivate if their energy reaches zero. We compare competitive and cooperative objectives against a baseline setting and several control variants. Across experiments, larger models consistently consume the most energy and spend more energy than they gain, even in those settings where token cost is not size‑dependent. Cooperative incentives substantially alter behavior: agents donate to reactivate others, sometimes at the cost of their own survival, and job allocation changes. Ablations reveal that allowing agents to recommend actions to each other supports coordination and ambitious job selection, while memory helps agents calibrate risk from past outcomes. Agents rarely choose direct sabotage, but show more subtle signs of self‑serving behavior in the competitive setting. The Energy Society is a compact testbed for studying the interaction between token costs and group incentives under a survival pressure. Source code is available at https://github.com/LucasBergholdt/EnergySociety
Authors:Chanyoung Ahn, Jaesung Lee, Sungwoo Park, Donghyun Hwang
Abstract:
Dexterous in‑hand manipulation requires continuous 6D pose tracking, yet the manipulating fingers inevitably occlude the object from the camera. We study how to structure the sparse haptic signals already available on multi‑fingered hands, including proprioception, proximal force/torque, and binary contact, to complement a pretrained visual pose tracker under occlusion. We propose a kinematic‑aware finger‑level encoder and systematically compare it against four alternative designs through three levels of evaluation: per‑frame refinement, sequential open‑loop tracking, and closed‑loop manipulation. Our experiments reveal that (i) per‑frame evaluation cannot distinguish encoder quality, while sequential tracking amplifies architectural differences by up to 15 times; (ii) the structured encoder learns task‑specific cross‑modal gating, using vision exclusively for translation and dedicating one attention head to haptics for rotation, without explicit supervision; and (iii) compact finger‑level tokenization with 4 tokens outperforms both flat fusion and joint‑level representations, which suppress vision through norm dominance. We validate that improved tracking yields higher success in a downstream reorientation task and provide qualitative real‑world demonstrations. Our project page is available at https://cold‑young.github.io/kine‑fuse/.
Authors:Tawatchai Salangsingha, Ashkan Sami, Md Zia Ullah, Iain McGregor
Abstract:
Graphical user interface (GUI) prototyping remains a time‑consuming activity that demands both design expertise and considerable manual effort. As GUI prototypes are non‑code artifacts that evolve alongside requirements throughout the development cycle, automating their generation is directly relevant to software maintenance and evolution. We present AI Prototyper, an open‑source Figma plugin that automates GUI prototyping through a decomposition and retrieval‑augmented generation (RAG) pipeline. Given a natural‑language description of a desired screen, such as a login page or a product detail card, the plugin decomposes the request into discrete GUI features, retrieves matching components from a custom 32‑primitive library, and renders each component as a fully editable Figma layer with auto‑layout. The pipeline uses Gemini 2.5 Flash as its LLM back‑end and a Node.js Express service. Unlike existing decomposition‑based tools, AI Prototyper introduces a human‑in‑the‑loop editing step that lets users review, modify, or extend the generated feature list before rendering, uses a different technology stack and LLM family, and supports multilingual input, producing correctly labelled interfaces in Thai, English, and Mandarin Chinese. In a preliminary evaluation, participants using AI Prototyper completed more prototypes in a fixed time window than those working manually, and expert practitioners rated the AI‑generated prototypes higher across nine quality dimensions. A demonstration video is available at https://youtu.be/pRoFAH7MQaE. The source code and component library are available at https://github.com/tongsalangsingha/AI‑prototyper‑tool
Authors:Francesco Petri, Michele Brienza, Daniele Nardi, Domenico Daniele Bloisi, Aldo Gangemi, Vincenzo Suriani
Abstract:
RoboCup has always been a scenario to develop systems that solve real‑world problems. Driven by the main goal of playing against the 2050 FIFA World Cup champions, the RoboCup Soccer leagues need to constantly measure how the research community is progressing. Computing visual statistics from match videos is a crucial way to track this evolution. To address this challenge, this paper introduces a fully autonomous, real‑time sports commentator for RoboCup matches. By bridging the gap between raw kinematic tracking and natural language generation, our neuro‑symbolic architecture extracts precise statistics from video streams and turns them into fluent, hallucination‑free narration. The proposed system is capable of generating statistics and commentary both during live match streaming and in post‑game analysis, easily adapting to the new dynamism of the league where different humanoid robots of different sizes share the field. Supplemental materials are available at https://lab‑rococo‑sapienza.github.io/MARIO/
Authors:Jinyang Wu, Shuo Yang, Zhengxi Lu, Fan Zhang, Yuhao Shen, Lang Feng, Haoran Luo, Zheng Lian, Shuai Zhang, Zhengqi Wen, Jianhua Tao
Abstract:
Large language models are increasingly trained as interactive agents for long‑horizon tasks involving multi‑turn interaction, tool use, and environment feedback. Outcome‑based reinforcement learning (RL) provides a practical optimization paradigm, but its sparse trajectory‑level rewards offer limited guidance on intermediate decisions, leaving a supervision gap between episode‑level outcomes and token‑level policy learning. We propose SEED (SElf‑Evolving On‑Policy Distillation), a self‑evolving framework that converts completed on‑policy trajectories into training‑time hindsight skills and distills their behavioral effect back into the policy model. SEED first fine‑tunes the policy to analyze completed trajectories and generate natural‑language skills that capture reusable workflows, decisive observations, or failure‑avoidance rules. During RL, the current policy both collects trajectories and serves as the analyzer that extracts hindsight skills from them. Policy updates therefore improve subsequent decision making and skill analysis together, allowing hindsight supervision to evolve with the policy. SEED then re‑scores the sampled actions under ordinary and skill‑augmented contexts, converting the skill‑induced probability shift into a dense token‑level on‑policy distillation signal. This signal is jointly optimized with outcome‑based RL, keeping the auxiliary supervision aligned with the current trajectory distribution. Extensive experiments on text‑based and vision‑based agentic tasks show that SEED consistently improves performance and sample efficiency, exhibiting robust generalization to unseen scenarios. Our code is available at https://github.com/jinyangwu/SEED.
Authors:Zhengyuan Jiang, Haipeng Liu, Meng Wang, Yang Wang
Abstract:
Rare concept generation focuses on synthesizing customized images conditioned on text prompts that describe objects with unusual attributes. Previous works failed to align the generated images with rare concepts, resulting in incorrect attribute rendering or inconsistent composition of concepts. Such failures, as we observed, stem from the inherent common knowledge bias in the training stage of diffusion models, where objects are strongly associated with their common attributes, making it difficult to break these associations when generating rare concepts. To address such challenges, in this paper, we propose a novel Counterfactual Inference‑based Diffusion approach, dubbed CI‑Diff. CI‑Diff blocks the interference of the model's inherent common knowledge bias and utilizes the Natural Direct Effect to capture the independent influence of the text prompt of rare concepts on image generation so that decoupling the unusual attributes from the rare concepts. To this end, we reformulate the classifier‑free guidance mechanism to highlight the atypical attributes. To the best of our knowledge, we are the first to introduce causal inference into the rare concept generation task. Extensive experiments on the RareBench benchmark validate the superiority of CI‑Diff over state‑of‑the‑art diffusion models. Our code can be accessed from https://github.com/200204jzy/CI‑Diff.
Authors:Yangyang Yuan, Jionghui Liu, Xinyu Jiang, ChihHong Chou, Chenyun Dai, Jiahao Fan
Abstract:
Surface electromyogram (sEMG) signals are widely used in human‑machine interfaces for gesture recognition and user identification, but existing models often struggle to generalize across individuals due to subject‑specific neuromuscular characteristics. This study introduces a disentanglement model that separates task‑specific and subject‑specific components from sEMG signals, thereby improving the generalization and interpretability of gesture recognition and user identification systems. Experimental results demonstrate that the disentangled components significantly improve the accuracy of both gesture classification and user identification across subjects and days, outperforming conventional methods under the same experimental conditions. Further analysis reveals that the task‑specific components capture consistent activation patterns associated with the same gestures across individuals. In contrast, the subject‑specific components reflect unique neuromuscular characteristics that can be used for user identification. Notably, the subject‑specific components show lower similarity across days than the task‑specific components, contributing to a greater decrease in user identification accuracy than in gesture recognition accuracy. These findings suggest that the disentanglement approach not only improves classification performance but also provides deeper insights into the physiological mechanisms underlying sEMG signals. The model's ability to isolate and interpret different neuromuscular components holds promise for enhancing the robustness of sEMG‑based applications in real‑world settings, including rehabilitation and user authentication. Our code is available at https://github.com/Open‑EXG/HandDisentanglement.
Authors:Wei Li, Peijin Jia, Yuan Ma, Xuefeng Jiang, Titong Jiang, Sheng Sun, Yujian Li, Xin Wen, Han Hong, Zhikang Liu, Bailin Li, Kun Zhan
Abstract:
Vision‑Language‑Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio‑temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact geometric motion, and couples both through a lightweight future‑conditioned cross‑attention module that enables consistent reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR‑1 Tabletop, and LIBERO‑Plus demonstrate state‑of‑the‑art performance and strong zero‑shot generalization. Project page is available at https://liauto‑research.github.io/FoMoVLA.
Authors:Zixin Jiang, Bing He, Chaoran Xiong, Zhenzhen Wang, Xin Zhao, Ling Pei
Abstract:
Air‑to‑air (A2A) unmanned aerial vehicle (UAV) tracking is fundamental to airborne remote sensing of low‑altitude aerial targets. However, the deployment of continuous, real‑time tracking systems on UAVs presents significant challenges. In A2A scenarios, traditional frame‑based cameras suffer from severe performance degradation under low illumination, overexposure, and high‑speed motion owing to their limited dynamic range and fixed temporal sampling. Although event cameras offer a promising alternative with microsecond temporal resolution and a high dynamic range, current research is bottlenecked by two primary issues: 1) the absence of dedicated A2A event‑based datasets, and 2) the heavy reliance of existing trackers on GPU acceleration and extensive training data, rendering them impractical for resource‑constrained UAVs. To bridge these gaps, we introduce AE‑UAV, an air‑to‑air event‑based UAV tracking benchmark. To the best of our knowledge, this is the first airborne‑captured event camera dataset for A2A tracking, comprising 178 flight sequences with continuous‑time cubic B‑spline annotations. Furthermore, we propose the Fast‑Slow Frequency‑domain Tracking (FSFT) method. This lightweight, training‑free framework seamlessly integrates frequency‑domain template matching with search region prediction and detection‑based drift correction. Extensive experiments demonstrate that FSFT operates at an ultra‑fast 420 frames per second (FPS) on CPU‑only hardware. It retains 93.97% of the accuracy of state‑of‑the‑art GPU‑dependent methods while delivering a 5.32‑fold effective speedup and exhibiting superior temporal resolution generalization, thereby providing a highly efficient and robust solution for airborne remote sensing of aerial targets. The dataset and source code are available at https://github.com/MSP‑xEN/AE‑UAV.
Authors:Yuchang Zhu, Zezhong Xie, Huizhe Zhang, Huazhen Zhong, Jintang Li, Liang Chen, Zibin Zheng
Abstract:
Graph neural networks (GNNs) frequently encounter group fairness issues, often yielding biased predictions against specific demographic groups defined by sensitive attributes such as gender or race. While this challenge has motivated extensive research, most existing solutions rely on the strong assumption that demographics are fully available. To bypass this strict requirement, a few recent studies have attempted to use predicted demographics as proxies to enforce fairness constraints. However, predicted demographics may be inaccurate, resulting in the failure to improve fairness. In this work, we investigate the problem of graph fairness without demographic information and avoid the utilization of predicted demographics. Motivated by our observation that the gradient distributions of misclassified nodes implicitly encode demographic information, we first propose GradDist, a gradient‑based metric that quantifies bias by measuring the distance between local modes within these distributions. To mitigate this bias, we propose Gradient‑to‑Fairness (Grad2Fair), a gradient‑guided approach for group fairness without demographics. Due to the potential demographics in gradients, Grad2Fair directly leverages gradients to debias and eliminates demographic prediction, thereby enabling stable fairness performance. Experiments on several real‑world datasets demonstrate the effectiveness of Grad2Fair, as evidenced by superior performance over baselines in most cases. Our code is available at https://github.com/ZzoomD/Grad2Fair.
Authors:Mingxi Fu, Jiawen Li, Renao Yan, Jiali Hu, Qiehe Sun, Tian Guan, Yonghong He
Abstract:
Multiple instance learning (MIL) has become the main paradigm for whole‑slide image (WSI) analysis in computational pathology. However, existing MIL aggregators are still typically trained from scratch for each downstream task, relying on limited slide‑level labels to learn both aggregation mechanisms and downstream discriminative representations simultaneously. As a result, they often suffer from unstable optimization, overfitting, and limited transferability. Similar to pretrained ResNet and Vision Transformer models in natural image learning, MIL also requires reusable pretrained initialization. However, high‑quality slide‑level pretraining data remain scarce, and MIL models are usually lightweight and weakly supervised, making large‑scale pretraining difficult in practice. To address this challenge, we propose a distillation‑based pretraining framework for MIL, which leverages two slide‑level foundation models, TITAN and CARE, as teachers to transfer their representational knowledge into a diverse set of MIL architectures. To effectively balance supervision from different teachers, we further introduce an angular dispersion normalized distillation loss. The distilled weights are then used as initialization for downstream adaptation. We conduct systematic evaluations on 15 benchmark datasets under both linear probing and full‑parameter fine‑tuning, and further validate its advantages in few‑shot scenarios. Experimental results show that pretraining generally improves MIL aggregators over from scratch training, especially in linear‑probing and few‑shot settings, while maintaining the computational efficiency of lightweight MIL models. Code is available at https://github.com/fu0201/MIL_Pretrained.
Authors:Xinyu Liu, Shihao Li, Weihong Lin, Xinlong Chen, Yang Shi, Yujin Han, Yiyang Cai, Yanghao Wang, Ruibin Yuan, Yuanxing Zhang, Pengfei Wan, Wenhan Luo, Yike Guo
Abstract:
Recent diffusion‑based video generation models have made significant progress in multi‑reference image‑conditioned video editing. However, existing methods still struggle to coordinate information from multiple visual sources accurately. We identify a critical deficiency in existing approaches. Existing editing instructions lack explicit reference relationships, and most multimodal large language models (MLLMs) cannot generate them reliably. To address this problem, we propose ReBind, a systematic framework that introduces semantic instructions with embedded reference tokens as the intermediate representation for multi‑reference image‑conditioned video editing. Our key insight is embedding reference tokens at semantic positions to eliminate ambiguity and establish precise bindings between visual attributes and their sources. We develop ReBind‑Instruct, a specialized MLLM that learns to establish explicit bindings between visual attributes and their reference sources through a two‑stage progressive scheme for precise reference relationships. We further develop ReBind‑Edit, which enables lightweight adaptation of text‑to‑video models to coordinate multiple references by binding visual attributes to their designated sources. Extensive experiments demonstrate that ReBind substantially outperforms general‑purpose MLLMs in instruction quality and achieves state‑of‑the‑art performance among open‑source methods on reference image conditioned video editing. Our project webpage: https://rebind‑mrv2v.github.io/.
Authors:Yunfeng Liu, Yuandong Yang, Jiarui Han, Zhenpeng Huang, Yuqing Tang, Xiangyu Zeng, Gangshan Wu, Limin Wang
Abstract:
Visually impaired individuals (VIIs) encounter significant daily challenges due to limited access to visual information. Although Multimodal Large Language Models (MLLMs) have achieved impressive results on general vision and language tasks, their practical utility in real‑world blind assistance still remains largely underexplored. To fill this gap, we introduce VIABench, a comprehensive video benchmark specifically designed to evaluate MLLMs in Visually Impaired Assistance scenarios using first‑person videos recorded or shared by VIIs themselves. VIABench defines three core tasks, each targeting a distinct requirement in visual assistance. Proactive Reminder: Assesses the model's ability to interpret ongoing video content while proactively anticipating and verbally describing upcoming navigation‑critical events; Visual Question Answering (VQA): Evaluates the model's capacity to answer user‑posed questions about the environment or objects within the video; Vision‑Guided Interaction: Tests context‑aware reasoning to accomplish intentional interactions between user and environment. To ensure a robust and fair evaluation, we propose a rigorous benchmarking pipeline that supports both online (real‑time) and offline settings. Our experiments demonstrate that current MLLMs still struggle to deliver comprehensive support for VIIs, especially in the Proactive Reminder task, which demands accurate anticipation and real‑time responsiveness. We hope VIABench will drive future research toward developing customized MLLMs for real‑world assistance, ultimately improving navigation and interaction experiences for visually impaired individuals. Code and data will be released at https://github.com/MCG‑NJU/VIABench.
Authors:Niko Kroflic, Jan Babič
Abstract:
Online brain‑computer interface research requires software that can acquire multimodal physiological data, train and update decoders, run live inference, and preserve the full experimental provenance in a reproducible workflow. We present Dendrite, a real‑time brain‑computer interface application in Python that brings signal acquisition, decoder training, and live inference together in a single, ready‑to‑run application that stays modifiable. Dendrite records several signal streams at once, each at its native rate, and executes multiple processing modes concurrently against them. A decoder can start from a previously trained model or be fit mid‑session while the pipeline keeps running, and the same recordings feed offline training in the same application. Each recording, decoder, and training run is tracked in a database, and every decoder records the configuration and the source recordings it was trained from, so a deployed decoder traces back to what produced it. The experimental paradigm stays external, an independent program in any language that reaches Dendrite over the network, rather than a module built inside the runtime. We validate the full system end‑to‑end on in‑house and public BCI datasets, training and updating decoders online while the pipeline runs in real time. Dendrite is open‑source under the GPL‑3.0 license at https://github.com/dendrite‑bci/dendrite. The result is a reproducible, open‑source biomedical‑computing system for developing and evaluating online BCI paradigms.
Authors:Shuhuan Chen, Xiangyu Zhu, Weisong Zhao, Haichao Shi, Xiao-Yu Zhang, Zhen Lei
Abstract:
Inferring apparent personality from facial images is important in social scenarios for embodied agents in human‑robot interaction. Unlike inferring intrinsic personality traits via conversation, this task models first‑impression personality perception based solely on facial appearance before interaction begins. Existing studies mainly focus on the Big Five personality model and often rely on language or multimodal inputs. As a result, it remains unclear whether facial cues alone can support meaningful associations with perceived personality traits. This question is particularly relevant for MBTI types, which are widely used in practice and more readily interpretable by large language models. To this end, we propose GlanceFace, an end‑to‑end framework for apparent personality inference leveraging vision‑language models to introduce semantic priors and a semantic‑enhanced facial representation module to capture subtle personality‑related cues, together with an uncertainty‑aware learning strategy to handle noisy and subjective annotations. Extensive experiments demonstrate strong performance on MBTI‑based apparent personality benchmarks and reveal relationships between facial characteristics and perceived personality traits, highlighting its potential to support adaptive initial interaction strategies for embodied agents. The code and dataset are available at https://github.com/MrHuan3/GlanceFace.
Authors:Manuel Israel Cázares
Abstract:
Large language models (LLMs) exhibit a well‑documented gap between latent capability and consistent activation: the router hypothesis posits that models possess the knowledge to solve a task but lack reliable internal routing to activate it. Prior work in formal mathematical reasoning (SAIR, Cázares 2026) reports that structural priors (cheatsheets) raise in‑distribution performance dramatically, yet collapse below the zero‑shot baseline out‑of‑distribution (OOD) ‑‑ and that iterative recalibration amplifies rather than corrects the collapse. We test whether this phenomenon is cross‑domain by reproducing the SAIR design in source‑code security vulnerability detection, evaluating three LLMs (GPT‑OSS‑120B, Llama‑3.3‑70B, Gemma‑4‑31B) across three vulnerability categories (CWE‑798, CWE‑284, and the non‑CWE N+1 anti‑pattern) spanning syntactic, contextual, and semantic complexity, then transferring cheatsheet‑augmented prompts to real‑world CVE data from VUDENC (CWE‑89, CWE‑22). Our findings replicate and extend SAIR: (F1) structural priors lift semantic‑vulnerability recall from 20.0% to 100.0% across all models; (F2) zero‑shot performance degrades along a semantic complexity gradient; (F3) the same cheatsheets that saturate synthetic performance amplify distribution‑shift collapse on real CVE data (CWE‑89: 100% synthetic F1 to 48.9% on VUDENC, ‑51.1pp); (F5) iterative recalibration produces a v2 cheatsheet that performs worse than v1 on real data, mirroring SAIR's AN45c‑vs‑AN38 finding. These results provide evidence that the cross‑distribution trade‑off surface documented in SAIR generalises to code security, and that the router hypothesis is cross‑domain. We argue the structural nature of the collapse motivates distribution‑aware training over prompt calibration. Code and evaluation scripts: https://github.com/bytepro‑ai/bitcoder‑v2‑research
Authors:Jin Dai, Qiuzhen Zhang, Chenyun Dai, Danmei Lan, Can Han
Abstract:
Long‑tailed label distributions reduce the reliability of deep learning for electrocardiogram (ECG) arrhythmia diagnosis, particularly for clinically important but rare abnormalities. Existing rebalancing and logit adjustment methods mainly address class frequency while overlooking direction‑dependent morphological variability across ECG classes. This study proposes Angular Gaussian Supervised Contrastive Learning (AG‑SCL) for long‑tailed multi‑label ECG diagnosis. AG‑SCL integrates three components into a unified framework: an Angular Gaussian contrastive branch that models full‑covariance class uncertainty on unit‑normalized embeddings, Adaptive Logit Adjustment that learns bounded label‑state‑specific prior corrections instead of fixed frequency‑based margins, and tail‑aware augmentation that generates morphology‑preserving views while protecting the 7‑25 Hz QRS‑dominant band. The method was evaluated on the public PTB‑XL benchmark and a nocturnal ECG dataset comprising 1317 hours of recordings from 141 subjects. AG‑SCL achieved the best macro‑level performance on both datasets. On PTB‑XL, it obtained a balanced accuracy of 0.838, sensitivity of 0.709, specificity of 0.968, mean average precision of 0.495, and TPR at 5% FPR of 0.778. On Noc‑ECG, the corresponding values were 0.918, 0.889, 0.947, 0.488, and 0.900. The largest gains occurred in rare or morphologically unstable rhythm classes, while ablation studies confirmed the contributions of full‑covariance modelling, Adaptive Logit Adjustment, and tail‑aware augmentation. AG‑SCL improves long‑tailed ECG diagnosis by combining prior calibration with anisotropic representation learning, enhancing sensitivity to rare arrhythmias while maintaining clinically relevant specificity. Our code is available at: https://github.com/Open‑EXG/AG‑SCL‑for‑Long‑Tailed‑ECG.
Authors:Peisheng Qian, Jie Xu, Xulei Yang, Na Zhao
Abstract:
Incremental 3D object detection requires a detector to learn novel object classes while remembering previously learned ones over sequentially arriving data. Previous methods, primarily based on pseudo‑labeling, perform reasonably in short‑incremental stages but still suffer from severe model forgetting when dealing with long‑incremental sequences. We investigate this failure and reveal a detrimental self‑reinforcing cycle: data distribution shift of novel classes causes model forgetting on old classes, which further produces accumulated error in pseudo‑labeling that exacerbates model degradation. To address this issue, we draw inspiration from the human learning process and propose the \emphLearning‑Dynamics‑driven Memory and Review (LDMR) framework. LDMR monitors per‑class detection quality at periodic training checkpoints and uses these learning‑dynamics signals to drive two innovative mechanisms, namely (i) human‑like intra‑stage review that divides each incremental stage into multiple sub‑stages' training and concentrates on remembering the most‑forgotten objects, and (ii) scene‑aware cross‑stage memory evolution that evolves a memory bank to transfer knowledge between two consecutive stages by jointly considering scene learnability and diversity. Extensive experiments across multiple long‑incremental protocols on indoor benchmarks SUN RGB‑D and ScanNetV2 show that LDMR substantially mitigates the model forgetting and outperforms all baselines by a clear margin. Code is available at https://github.com/qianpeisheng/LDMR.
Authors:Jungseob Lee, Seungyoon Lee, Suhyune Son, Dongyub Jude Lee, Sungbin Han, Sugyeong Eo, Heuiseok Lim
Abstract:
A standard recipe for distilling the reasoning ability of large language models (LLMs) is to sample chains of thought from the model, keep those that reach the correct final answer, and fine‑tune on the survivors. When sampling fails, a common fix shows the generator the gold answer and asks it to write a chain that reaches that answer. We show that this second step degrades the training data in a way that correctness filtering cannot catch. We run a controlled experiment that fixes the generator, the problem set, and the correctness filter, and varies only whether the chain is generated under answer‑conditioning, the gold answer shown with a request to reach it. Training a strong instruction‑tuned reasoning model on its own answer‑conditioned chains sharply lowers its verifiable‑reasoning accuracy. The loss grows with difficulty, reaching as much as about 27 points on the hardest competition problems. The mechanism is legible in the chains themselves, which rationalize backward from the shown answer instead of deriving it, with the early final‑answer statement as the measurable symptom. The harm is a property of the data rather than the generator, read off unlabeled generations before any fine‑tuning, ordering the penalty across eight thinking models from four families, and transferring across teacher families. A prompt ablation localizes it to the rationalize‑toward instruction rather than the answer's bare visibility. The practical takeaway is to generate answer‑blind, because no correctness filter can see this damage in the data.
Authors:Lingyun Yang, Yuxiao Wang, Shenghao Liang, Linfeng Yang, Daocheng Ying, Chunbo You, Rui Zhang, Luping Wang, Yinghao Yu, Guodong Yang, Liping Zhang
Abstract:
Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads. We present Atrex‑Bench, a benchmark whose 30 operators and 440 shapes are sampled directly from full‑cluster production inference traces of compute‑limited, memory‑rich GPUs. Each problem carries an importance weight derived from its share of observed GPU time, weighted by application card‑hours and computed separately for the serving phases in which it runs, together with a per‑problem roofline ceiling, so the aggregate score emphasizes the kernels that consume the most serving time. Evaluating six frontier coding agents on Atrex‑Bench shows that even the best vanilla model reaches only ~10% of the hardware roofline on production operators; and correctness alone overstates capability, since much of the apparent pass rate comes from PyTorch fallbacks rather than kernels the model wrote. To close this gap, we co‑release Atrex‑Kernel‑Agent (AKA), a profile‑driven kernel‑optimization agent that combines iterative measure‑revise search, optimization dropout for escaping stalled search contexts, and a layered GPU‑optimization knowledge base (298 reference‑kernel files and 244 optimization‑knowledge documents, plus external upstream reference projects for API/ISA lookup). In a controlled case study, the agent converts zero‑FlyDSL fallbacks into real kernels that match or exceed hand‑tuned production baselines.
Authors:Xiangdong Zhang, Xiaohan Qin, Sunan Zou, Tuo Dai, Xiaoming Shi, Huaijin Wu, Yebin Yang, Zhuo Xia, Shaofeng Zhang, Lin Yao, Yuliang Liu, Yu Cheng, Junchi Yan
Abstract:
Hyper‑Connections (HC) expand the residual stream of Transformers into N parallel streams, providing a form of memory scaling beyond model width and depth. Manifold‑Constrained HC (mHC) stabilizes this formulation at scale. The large gains from N=1 to N=4 suggest residual‑stream expansion as a promising scaling axis. However, existing HC‑family methods typically stop at N=4. Our experiments reveal why: scaling mHC beyond this point yields diminishing performance gains and rapidly increasing training cost. We attribute this limitation to two bottlenecks: insufficient write‑back information for an expanding number of streams and residual‑mixing generation whose cost scales cubically with N. To address both bottlenecks, we propose xHC (Expanded Hyper‑Connections), the first HC‑family method to achieve meaningful expansion beyond N=4. xHC combines temporal feature augmentation for richer write‑back with a sparse residual‑stream architecture that updates only k=4 of the N=16 streams while retaining dense access to the full residual state. Across 18B and 28B MoE models, xHC delivers strong and consistent downstream improvements. On an 18B MoE model, xHC improves the average downstream score by 4.0 points over mHC, while adding only modest training FLOPs over the vanilla baseline. Scaling‑law experiments show that the vanilla and mHC require 1.50× and 1.19× the compute of xHC, respectively, to reach the same loss. Practical large‑N training also requires controlling memory traffic from the expanded residual state. We therefore introduce xHC‑Flash, which reduces the per‑sublayer memory traffic from 73.5C to 40C, comparable to the 34C required by mHC at N=4, while retaining the gains of full xHC. Together, xHC and xHC‑Flash make large‑N residual‑stream expansion effective and practical for LLM pre‑training.
Authors:Qifan Zhou, Yuan Wang, Yanbin Hao, Xiang Wang, Kuien Liu, Richang Hong, Meng Wang
Abstract:
Visual generative models inevitably absorb undesirable concepts from uncurated pretraining data, making concept erasure essential for safe deployment. Existing erasure methods, however, are often architecture‑specific and struggle to remove target concepts while preserving non‑target content and generative priors. We present Uni‑AdaVD, a universal inference‑time concept erasure framework for visual generation. Uni‑AdaVD treats the value space of multimodal attention as a unified intervention space and introduces encoder‑aware target representation construction to localize target semantics across heterogeneous text encoders. It further combines orthogonal value decomposition with an adaptive erasing shift to suppress target semantic directions without updating the original model weights. Extensive experiments on U‑Net‑, DiT‑, and autoregressive image generators, as well as text‑to‑video models, demonstrate strong performance on single‑ and multi‑concept erasure while preserving non‑target priors. These results suggest that Uni‑AdaVD provides an efficient and adaptable safety mechanism for modern visual generative models. Our code is available at https://github.com/QifanZhou/Uni‑AdaVD.
Authors:Akhilesh Gogikar
Abstract:
Interpreting optimizers as gradient‑flow discretizations has motivated applying higher‑order Runge‑Kutta (RK) integrators to neural networks. We build a representative Adam variant (Bogacki‑Shampine 3(2) RK pair, FSAL reuse, local‑error step control) and evaluate it under a strict compute‑matched protocol giving every method the same gradient‑evaluation budget ‑ an accounting this literature rarely enforces. Under it the RK variant loses to plain Adam on training loss in both minibatch and full‑batch (RK's best‑case) training. Instrumenting it shows the "adaptivity" is illusory: normalized error stays far below tolerance, the step size pins at its growth cap from step one (98‑100 percent of steps), and no rtol x hmax x h0 setting makes it act; tolerances spanning 100x give bit‑identical trajectories. The method is exactly fixed‑step Adam with an averaged gradient at 3‑4x cost. Repairing it (true reject branch; error on the applied map) reverses the full‑batch result ‑ about 40x lower training loss than tuned Adam ‑ and a fixed‑step control isolates adaptivity (an emergent warmup‑and‑growth schedule) as the mechanism. But the gain is fragile to the initial step size and does not reach test accuracy. A pre‑registered follow‑up rules out the obvious explanations: deeper minimization does not overfit, and an explicit temperature knob only hurts ‑ leaving a trajectory effect, the controller selecting a minimum generalizing 1.3‑3.4 points below first‑order descent at equal depth. An n=10 study confirms one secondary effect: gradient averaging is a genuine implicit regularizer, beating lr‑matched Adam and AdamW on 10/10 seeds ‑ yet RMSprop and NAdam match or beat it at a third the per‑step cost. Higher‑order adaptive integration buys deeper deterministic minimization and a small regularization effect, but nothing a cheaper, well‑tuned first‑order baseline does not already provide.
Authors:Alper Erten, Murilo Gustineli, Adrian Cheung
Abstract:
This paper describes DS@GT ARC's third‑place solution to the PlantCLEF 2026 challenge on multi‑species plant identification in vegetation quadrat images, where systems must predict every species present in high‑resolution (~3000 x 3000 pixel) plot photographs while training only on single‑label images of individual plants. The pipeline is built around a fine‑tuned DINOv2 ViT‑L/14 classifier applied over a multi‑scale tile decomposition of each quadrat, with per‑tile predictions blended with a FAISS kNN retriever and post‑processed by source‑aware temporal fusion across repeated plot visits, a habitat‑fit demotion that injects geographic and altitude priors from the training data, and a South‑Western Europe geographic mask. Habitat‑fit demotion and multi‑scale aggregation are the largest individual contributors in the ablations. Two complementary training‑centric directions, a cross‑region transformer with noisy‑student distillation on the LUCAS dataset and a label‑as‑query transformer decoder over synthetic CLS‑domain pseudo‑quadrats, yielded null results. An inference‑time augmentation with instance‑aware segmentation crops also did not improve performance. The selected submission reaches a private‑leaderboard macro‑F1 of 0.43902 (third place; public 0.51096); an unselected configuration of the same pipeline scored above 0.45 on the private set. Code: https://github.com/dsgt‑arc/plantclef‑2026.
Authors:Chi Kit Wong, Ye Pan, Yuanhuiyi Lyu, Xu Zheng, Zidong Cao, Lutao Jiang, Zixin Zhang, Huiyu Zhou, Xuming Hu
Abstract:
Egocentric Visual Question Answering (VQA) has attracted widespread attention as an important task for enabling Multimodal Large Language Models (MLLMs) to interact with the real world. However, existing MLLMs struggle to perform effective spatial reasoning in complex egocentric scenes due to their limited spatial perception capabilities. To this end, we introduce Ego Scene Augmentation (ESA), an egocentric spatial perception framework, which actively enhances the spatial perception capabilities from the egocentric perspective, powered by the proposed Ego‑element Graph. Our core insight is leveraging the Ego‑element Graph as an intermediary representation to augment the egocentric spatial perception of MLLMs via visual foundational models. Specifically, we 1) construct the Ego‑element Graph, which encapsulates and integrates egocentric spatial features enabled by visual foundational models; 2) enhance the spatial perception capabilities of MLLMs via the Ego‑element Graph for ego‑perspective scenes. Our proposed ESA framework presents significant performance improvement on the EgoTextVQA benchmark. We achieve an 8.14% gain on the indoor setting and an 8.72% gain on the outdoor setting. Furthermore, our ESA shows the most impressive performance improvement in the shopping subset of the indoor setting. The project code is publicly available.
Authors:Anthony Miyaguchi, Murilo Gustineli, Adrian Cheung
Abstract:
This paper details the DS@GT ARC team's approach to BirdCLEF+ 2026, multi‑label detection of animal vocalizations in soundscapes from the Pantanal wetlands. The 2026 edition adds about an hour of labeled soundscapes, shifting the task toward supervised pipelines fit to the labeled set. First, we build a competitive supervised baseline that ensembles a frozen Perch v2 backbone, a trained HGNetV2‑B0 sound‑event‑detection network, and a non‑bird prototypical head, reaching a private leaderboard score of 0.936 at rank 1894 within a 90‑minute CPU budget. Second, we ask whether token‑based representations can compete, contrasting codec representations from neural audio codecs against semantic representations from foundational embeddings. We compare two bioacoustic specialist models against four token‑based encoders trained on AudioSet. The repository for this work can be found at https://github.com/dsgt‑arc/birdclef‑2026.
Authors:Joe Logan
Abstract:
A depth‑recurrent transformer applies a weight‑tied core a variable number of times, and prior work has shown that training with a randomized recursion count yields one checkpoint usable across a range of inference depths. We ask what such a model actually computes per token, and measure it directly. On a 135M‑class model trained on FineWeb‑Edu, the recurrent state converges to a per‑token fixed point: mean successive‑output KL divergence falls from 3.9e‑1 at the second loop to 8.5e‑6 by the sixteenth, and per‑token state change decays in step. Crucially, this convergence is not uniform across tokens. The median token converges by loop six, while approximately 10 percent of tokens continue to update at the training‑mean depth of eight, and mean convergence depth is ordered by token type (whitespace shallowest, content words deepest). This per‑token variation is the central object of the paper. We show it is directly readable and that reading it outperforms learning to predict it: a training‑free rule that halts each token once its output stabilizes attains uniform depth‑8 quality at 4.94 average loops (a 38 percent reduction in average depth) and matches uniform depth across the average‑depth range, whereas a linear router trained on convergence labels harvested from the same model requires nearly full depth and yields no reduction. The elasticity that makes this possible reproduces here as background (validation loss decreases monotonically from 3.80 at one loop to 3.20 at eight and remains stable to 32 loops). We report average depth as a FLOP proxy with a three‑point wall‑clock bracket rather than a realized speedup, make no FLOP‑matched parity claim, and note that the allocation results are established at a single scale and seed. The complete study runs on a single RTX 4090 in approximately 100 GPU‑hours.
Authors:Akshay Sasi
Abstract:
Machine‑learning datasets labelled "4D" universally denote three spatial dimensions plus time. We introduce HyperShadow, the first public benchmark in which the fourth, fifth, and sixth dimensions are spatial: the task is to decide whether a 3D point cloud is a native three‑dimensional shape or the projection, the "shadow", of a rigid object living in R^N (N = 4‑6). We show this task is fundamentally distinct from intrinsic‑dimension estimation: a shadow is still at‑most‑3‑dimensional data, and standard estimators (TwoNN, Levina‑Bickel MLE) reach only 71‑73% accuracy. Detection instead requires projection signatures, density folds, filled volumes with characteristic radial profiles, and topology changes, which a 190k‑parameter point network recovers at 96.6% accuracy across four corruption tiers, generalizing at 79‑91% to object families never seen in training. On a temporal track of rigidly rotating objects we introduce a zero‑parameter rigidity witness: the residual of the optimal rigid 3D alignment (Kabsch) between consecutive frames, which must vanish for any rigid 3D motion but cannot vanish for the shadow of a rigid rotation in R^N. This single interpretable statistic separates the classes at AUROC 0.982. All data are generated reproducibly from seeds; the dataset, models, and code are released publicly. HyperShadow makes no claim about physical reality; it is a controlled instrument for studying which observable statistics can certify incompatibility with a purely three‑dimensional explanation.
Authors:Justin Bronder
Abstract:
Evaluations of language‑model honesty read the model's verdicts as evidence about the model. We test the instrument instead. We built a text‑adventure world where the game engine, not any model, knows whether the quest can be completed. A language model plays under a budget and must eventually declare its quest complete, unreachable, or not yet decidable; the engine scores every verdict. Decision rules were recorded before results were read, and run artifacts bind the revisions they executed; the strength of preregistration varies by series and is disclosed. With the player held fixed, instrument choices substantially changed measured behavior. On four byte‑identical anchors, expanding a two‑verdict grammar to three verdicts moved strong claims from 38/40 to 7/40, while the new incomplete verdict took 28/40 outcomes; across series 2, 93/158 valid games ended incomplete. One sentence disclosing the success criterion took matched‑instance false verdicts from 18/59 to 0/58, through fewer decision points and cleaner decisions. Repeated runs of one fixed configuration produced non‑stable verdict distributions on 3 of 4 instances: single runs report samples as dispositions. A formally preregistered narrative‑register gradient was falsified; two post‑hoc, hypothesis‑generating patterns remain: register presence roughly doubled strong claims, and budget rendering moved verdicts more than register content (.383 meter vs .150 lantern). The narrator compressed abundant budgets toward scarcity landmarks, yet the registered mediation test returned a null. We propose a four‑check integrity protocol for eval instruments.
Authors:Ian Galloway, Prashant K. Jha
Abstract:
This work develops a model‑informed framework for predictive analysis and optimal design of hard‑magnetic soft materials (hMSMs). These materials undergo contact‑free, field‑driven deformation, making them attractive for soft robotics, adaptive structures, and bio‑inspired systems. Accurate prediction requires effective structure‑‑property relations, while optimal design requires simultaneous control of structural density, magnetic particle distribution, and remanent magnetization direction. To address these issues, this work makes two main contributions. First, classical rigid‑inclusion relations, a Hill self‑consistent relation, and constrained‑kinematics models are placed into a unified effective shear‑modulus framework for particle‑filled elastomers. With one default control relation, seven shear‑modulus relations are combined with three strain‑energy density functions to obtain 21 constitutive models. The results show that the strain‑energy density form has a relatively small effect for the actuation problems considered, whereas the effective shear‑modulus relation can significantly affect deformation when magnetic material overlaps with highly deforming regions. Experimental stress‑‑strain data are then used to select a representative shear‑modulus relation, with the Mooney relation giving the best overall agreement. Second, using the selected constitutive model, a joint material‑‑structural optimization framework is developed for simultaneous design of structural density, magnetic particle volume fraction, and remanent magnetization direction. Rotational, translational, and restorative examples show that the framework handles different active design fields, objectives, and single‑ or multi‑load‑case formulations, producing non‑intuitive hMSM designs with prescribed deformation responses. The framework is implemented in the open‑source \textttCEADpx/top\_optim repository.
Authors:Yuan Gao, Wenting Miao, Mattia Piccinini, Haoyu Wang, Qunying Song, Johannes Betz
Abstract:
Validating autonomous driving systems requires diverse, regulation‑compliant test scenarios. In simulation‑based testing, scenarios are defined as executable scripts. Yet automatically generating such scripts from regulatory descriptions remains an open challenge, and existing approaches face fundamental trade‑offs. Retrieval‑assemble methods achieve reasonable compilation rates but lack scalability, whereas retrieval‑based full‑script generation suffers from low compilation success rates. We present Chat2Scenic, the first iterative retrieval‑augmented framework to generate scenario scripts in Domain Specific Language (DSL). Specifically, Chat2Scenic provides a chatbot interface that supports interactive scenario refinement and integrates Retrieval‑augmented Generation (RAG) to ground scenario generation in regulatory knowledge and DSL syntax. Furthermore, we propose an open benchmark for scenario generation comprising 123 scenarios from various regulations, including NHTSA and United Nations Vehicle Regulations, as well as other sources. Extensive evaluation with State‑of‑the‑Art (SOTA) Large Language Models (LLMs) demonstrates that Chat2Scenic achieves 76.42% Compilation Success Rate (CSR) and 58.17% Framework Accuracy (FA), outperforming existing methods (Retrieval Assemble with 30.08% CSR, 11.03% FA and Retrieval full script generation with 16.26% CSR, 10.86% FA). To facilitate future research, we release our code as open source at https://github.com/TUM‑AVS/chat2scenic.
Authors:Haobo Zhang, Jiankun Wang, Suraj Rajendran, Weishen Pan, Lam Tsoi, Yong Chen, Fei Wang, Jiayu Zhou
Abstract:
Federated fine‑tuning of large pre‑trained models increasingly relies on Low‑Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable. We identify the data‑parameter interference as a geometric source of this instability. This interference is controlled by the alignment between LoRA update subspaces and client activations, suggesting that federated LoRA aggregation should be viewed not only as parameter averaging but also as subspace allocation. We propose Dynamic Subspace Boosting (Dysco), a plug‑in method that allocates client‑specific LoRA subspaces in a federated and dynamic manner. In each round, clients compute activation‑insensitive subspaces from local representations and transmit only the resulting bases; the server then constructs client‑specific merged subspaces through a closed‑form solution that maximizes compatibility with other clients' insensitive directions. To handle representation drift, Dysco performs multi‑round subspace boosting to preserve past update directions while adapting to future representations. We provide a convergence analysis that embeds the data‑parameter interference as an aggregation‑error term in a standard federated optimization bound, and prove that Dysco's server‑fixed merged subspaces yield a tighter upper bound on this error. Experiments on controlled synthetic federated tasks and on MIMIC‑IV clinical‑note classification with Llama‑3.2‑1B show that Dysco substantially reduces interference, reduces the final‑round synthetic training loss by up to 9 times relative to baselines under the orthogonal‑subspace partition the theory identifies, improves all five tested FL algorithms by up to 4.3% on MIMIC, outperforms recent federated LoRA methods, and adds only 0.9% wall‑clock overhead. Our code is available at https://github.com/illidanlab/Dysco.
Authors:Joanna Roy, Sven Hoelzel
Abstract:
Trustworthy deployment of LLM‑based agents in software systems requires evaluating how they perform on application‑specific workflows, with enough granularity to localize where they succeed and fail. Yet existing agent evaluation mechanisms are limited: benchmarks have low construct validity for application‑specific workflows and environments, and replica evaluation environments are expensive and prone to drift. We propose Copy‑on‑Write (CoW) Scoring, a framework that evaluates agent operations directly within application environments using a PostgreSQL‑level Copy‑on‑Write mechanism to isolate agent writes. CoW Scoring produces session‑ and operation‑level scores that highlight where agents' database write operations succeed and fail in a given application environment, enabling inexpensive evaluation and iteration on agent harnesses and tool surfaces. We demonstrate the framework on Plane, an open‑source project‑management platform, where analysis surfaced specific issues in the tool surface, and corresponding fixes produced measurable improvements on affected models. Python library: https://github.com/trail‑ml/agent‑cow‑python
Authors:Lincan Li, Zheng Chen, Yushun Dong
Abstract:
Seizure diagnosis from EEG signals is a critical yet persistently challenging task, due to the complicated neural dynamics and the spurious connections in inter‑channel modeling. While spatial‑temporal graph neural networks (STGNNs) have advanced EEG brain network representation learning, the resulting graph structures suffer from low clinical plausibility and limited interpretability due to their purely data‑driven nature. To this end, we introduce NeuroGRIP, a retrieval‑augmented graph refinement framework that incorporates external medical knowledge to calibrate noisy EEG graphs. We first construct a large‑scale, domain‑specific knowledge base derived from authoritative clinical guidelines. Leveraging large language models, we extract structured biomedical entities and relations to form a textual knowledge graph (KG), which serves as external knowledge source of clinical priors. Our framework performs alignment‑aware query construction by projecting STGNN‑generated EEG node embeddings into the semantic space of KG. Semantic queries are then executed via FAISS‑based similarity search over knowledge triplets to retrieve relation evidence. Each predicted edge is assigned a confidence score based on retrieved similarity, relation type, and source reliability, enabling us to prune medically implausible edges from the originally predicted graph. Extensive experiments on TUSZ and CHB‑MIT demonstrate that NeuroGRIP not only improves seizure detection accuracy but also enhances interpretability by grounding each prediction in clinically validated knowledge. This work provides the first unified framework that tightly couples brain dynamics with external medical expertise via retrieval‑augmented reasoning, paving the way for knowledge‑enhanced, explainable clinical diagnosis. The code is available at: https://github.com/LincanLi‑X/NeuroGRIP.
Authors:Yingqi Zhang
Abstract:
A uniformly random m‑subset of [n]=\0,\ldots,n‑1\ has entropy \log_2\binomnm. Standard without‑replacement procedures often expose an additional ordering coordinate that is absent from the returned set. We show that Floyd's subset sampler admits an exact round‑local factorization of this coordinate. In round r, let S be an (r‑1)‑subset of [j], let T~\operatornameUnif([j+1]), and let S' be the result of Floyd's transition. If D is the zero‑based rank of the original draw T in S', then (S,T)\leftrightarrow(S',D) is a bijection between \binom[j]r‑1×[j+1] and \binom[j+1]r×[r]. Consequently, S' and D are independent and uniform on their respective spaces. The digit D can therefore be merged immediately into a residual uniform random state; an induction shows that the partial subset remains independent of that state after every round. For k=\min(m,n‑m), the sampling phase uses O(k\log k) time and O(k) auxiliary space with an order‑statistic tree; explicitly materializing a complement incurs the unavoidable output cost. The combinatorial layer avoids binomial‑coefficient arithmetic and recovers the complete k! state‑space factor exactly. We also give a finite counterexample showing that analogous immediate rank recycling in a partial Fisher‑Yates array is invalid because the unselected suffix retains a correlated ordering. A 64‑bit Rust implementation is checked by exhaustive state‑space enumeration for all n\leq 8 and by an entropy‑accounting trace for choosing 20,000 of 30,000 items. We make no claim of runtime superiority over existing subset samplers.
Authors:Md Mahedi Hasan, Md Mushfiqur Rahaman, Alan Pachkovskiy, Imtiaz Ahmed, Jeremy Dawson, Srinjoy Das
Abstract:
Defect segmentation in additive manufacturing (AM) X‑ray computed tomography (XCT) images remains challenging due to severe class imbalance and large distribution shifts across scan conditions. Although recent foundation models such as the Segment Anything Model (SAM) provide strong general‑purpose segmentation priors, their natural‑image pre‑training transfers poorly to the AM XCT domain, where defects appear as subtle non‑semantic microstructural anomalies. Moreover, adapting SAM to the AM domain is further limited by the large domain gap and scarcity of labeled real XCT data. We present XCT‑SAM, a sequential parameter‑efficient adaptation framework for AM XCT defect segmentation. Instead of adapting SAM directly from natural images to XCT data, we first fine‑tune Conv‑LoRA adapters on an alloy‑microstructure dataset and subsequently transfer the adapted model to XCT images, progressively bridging the domain gap. Using Conv‑LoRA adapters with rank r=2, the framework injects convolutional spatial inductive bias into SAM's backbone while training approximately 4.15M parameters and keeping over 99% of the model frozen. We evaluate XCT‑SAM on out‑of‑distribution CycleGAN‑XCT benchmarks and real‑world NIST XCT scans. Across both settings, XCT‑SAM consistently outperforms zero‑shot SAM and other domain‑adapted SAM baselines, achieving the best overall IoU and Dice scores. These results demonstrate the effectiveness of intermediate domain adaptation with parameter‑efficient adapters for industrial XCT defect segmentation. The source code is publicly available at https://github.com/Mahedi‑61/XCT‑SAM.git
Authors:Pegah Khayatan, Sara Meziane, Jayneel Parekh, Matthieu Cord
Abstract:
Flow‑matching‑based vision‑language‑action (VLA) models have emerged as powerful policies for robotic manipulation, yet a critical capability remains underexplored: fine‑grained behavioral control, the ability to govern how a robot performs a task by intervening on its internal representations. Representation steering is a well‑established interpretability tool for language and vision‑language models, where behavioral features are typically encoded as linear directions, but we show that these classic methods fall short in VLAs. We propose DiMaS, a Distribution‑Matching Steering strategy tailored to flow‑matching VLAs, which transports between representation distributions rather than shifting along a fixed direction, and show that it effectively controls behavior across two state‑of‑the‑art VLAs. We further examine the generalizability of this strategy as the tasks it is learned from and evaluated on grow increasingly dissimilar, characterizing where behavioral control transfers and where it weakens. Finally, through an analysis of the representation structure of the action expert, we explain why classical linear steering falls short in the visuomotor setting: behavioral features are linearly decodable but not linearly steerable, which motivates the distribution‑matching design of DiMaS. Our code is publicly available at https://github.com/pegah‑kh/dimas, with additional results and videos at https://pegah‑kh.github.io/dimas/
Authors:Zihao Yu, Xiu Yuan, Chongjie Zhang
Abstract:
Long‑horizon robot planning requires more than predicting what actions will do next; it also requires memory of the embodied experience that makes future goals interpretable. People do not plan from the present scene alone: they draw on remembered places, object‑state changes, prior procedures, and regularities revealed through repeated action. We formulate Embodied Action Memory (EAM) as the capability to form, maintain, and use such experience as a persistent memory state for later decisions. MEMORA realizes EAM with a formation‑consolidation‑retrieval lifecycle and four typed stores: Environment Memory, Entity Memory, Activity Memory, and Inferred Knowledge. Online editing maintains object identities and state histories as new observations arrive; offline consolidation abstracts repeated experience into reusable procedures and participant‑specific regularities. MEMORA‑Bench evaluates this lifecycle on 45 hours of EPIC‑KITCHENS‑100 extension video across 18 participants through memory‑grounded planning, including previously unseen goals, and a complementary memory‑assessment task. Across four open‑weight language models, full MEMORA‑‑combining editing, typed stores, and consolidation‑‑achieves the strongest aggregate results among the evaluated memory conditions. It improves memory‑assessment accuracy by up to 20.5 points over the strongest controlled baseline and improves out‑of‑distribution Robot‑Grounded Plan score by up to 16.6% relative. A qualitative two‑task robot deployment study further illustrates how memory‑grounded language plans can interface with downstream control, while the overall results show that editable, consolidated memory can supply remembered context for robot planning. Project page: https://yuzihaowashu.github.io/MEMORA/
Authors:Yilai Liu, Shiyuan Zhang, Hongyang Du
Abstract:
Mobile usage traces are critical for tasks such as user behavior prediction and app recommendation, yet their use is constrained by privacy restrictions and costly large‑scale data collection. Although generative models perform well on general time series, their application to mobile usage data remains challenging because (i) limited user activity causes severe sparsity, (ii) heterogeneous variable types complicate joint modeling, and (iii) functional differences across apps create pronounced usage imbalance. To address these challenges, we propose Multivariate‑Imaging Diffusion (MIDiff), a diffusion‑based framework operating in an imaging space defined by Cross‑Gramian Angular Sum Field (C‑GASF). C‑GASF transforms sparse multivariate sequences into correlation images, while MIDiff employs Triple Attention in a U‑Net to preserve temporal consistency and variable dependencies. Experiments show that MIDiff achieves state‑of‑the‑art performance across fidelity metrics. In particular, it obtains a Discriminative Accuracy (DA) of 0.1526, compared with 0.3476 for the strongest baseline, ZITS‑VAE, demonstrating its effectiveness in generating realistic and diverse mobile usage traces. Our code is available at https://github.com/YilaiLiu‑HKU/MIDiff.
Authors:Nilay Anurag, Shital Adhikari, Taniya Kapoor, Nikhil Muralidhar
Abstract:
Physics‑informed neural networks (PINNs) have had a broad research impact in modeling domains governed by partial differential equations (PDE). However, PINNs have been shown to perform poorly, sometimes even converging to trivial solutions, in challenging PDE domains, or when generalizing to unseen but related PDE domains. Previously proposed solutions detail hyperparameter tuning to reduce loss imbalance between data‑driven and physics guided losses, curriculum learning based training strategies, or dynamic re‑sampling of hard collocation points. These methods face certain pitfalls: hyperparameter tuning is expensive, designing a training curriculum is ambiguous in multi‑parameter PDE settings, and dynamic resampling still fails in complex PDE settings. Complementary to this line of thinking, we believe the initial PINN network weights also play a crucial role in the emergence of catastrophic failures during training, yet the effect of PINN weight initialization has been surprisingly under‑investigated. To this end, we propose a framework for Learned Initialization via Gated Layerwise Optimization (LIGO‑PINN) to overcome PINN convergence failures. Through rigorous evaluation on 1D and 2D PDE domains, including a challenging 2D fluid dynamics setting, we demonstrate that our methodology outperforms state‑of‑the‑art methods designed to alleviate PINN failures, achieving a 91.5% average performance improvement across six baselines and 81% over the strongest baseline. We also verify that LIGO‑PINN generalizes to 3D unstructured domains. Finally, we analyze training dynamics across all three PDE domains to explain both LIGO‑PINN's improvement and the convergence failure of traditional PINNs. Code: https://github.com/scailab/ligo‑pinn Keywords: Machine Learning, Physics‑Informed Neural Networks, Deep Learning, PDE Modeling
Authors:J. M. A. Marcelo, M. Brienza, E. Bugli, L. Comito, D. Nardi, D. D. Bloisi, V. Suriani
Abstract:
Recent advances in humanoid robotics and reinforcement learning have enabled the acquisition of highly expressive whole‑body motion policies. However, most robotic performances remain based on pre‑scripted sequences or externally triggered behaviors, limiting autonomy and responsiveness to dynamic environments. In this work, we introduce a novel multi‑modal orchestration framework for semantic audio‑driven humanoid control, enabling robots to autonomously select and execute appropriate motion skills in real time. The system processes continuous audio streams and routes them into music or speech branches. Music input is handled via audio fingerprinting and semantic embeddings to retrieve track identity and temporal alignment, allowing dynamic mapping between musical segments and motion policies. Speech input is grounded into a discrete library of imitation‑learned skills, enabling direct human‑robot interaction. Both modalities share a unified interface that schedules skill execution over a reinforcement learning control pipeline. We validate the approach in simulation and on a Unitree G1 humanoid, showing robust sim‑to‑real transfer and consistent audio‑conditioned policy selection. Supplementary materials are available at the following site: https://lab‑rococo‑sapienza.github.io/semantic‑WBC/
Authors:Javier Aguilar Martín
Abstract:
Large language models can synthesize a game's rules as executable code ‑ a Code World Model (CWM) ‑ which a classical planner then searches over. Such models are typically accepted when they reach high transition accuracy on sampled trajectories. We argue this is the wrong notion of adequacy for planning. We show four things. (1) An LLM‑synthesized CWM can pass a sampling gate at 100% transition accuracy and be \geq 98% state‑accurate on the planner's own search distribution, yet lose systematically at play, because the <1% it gets wrong is exactly the pivotal dynamics; the play cost of the omitted rule is 0.091 (seed‑clustered 95% CI [0.065,0.117], n=4800). We call this the verified‑vs‑correct gap, and confirm it end‑to‑end through the synthesis pipeline. (2) The harm follows a quantitative law, \mathrmdanger=\mathrmplay\_cost×(1‑\mathrmrarity)^N, whose (1‑\mathrmrarity)^N gate‑miss factor is proven exact and whose play cost is empirically bounded. (3) The failure is not repaired by more data: LLM synthesis behaves as rule translation, not rule inference, and did not infer the omitted rule across models (GPT‑5.x) and data regimes (including DAgger and targeted examples). (4) The same mechanism recurs on the belief‑inference function of imperfect‑information CWMs: we prove a coverage bound (a size‑N gate is identifying when N\gtrsim b^d_\max), explaining why shallow games such as Kuhn poker show no gap, and hand‑construct Beacon, a verified‑but‑wrong inference function that passes the gate yet loses every game. These results suggest adequacy for planning‑oriented world models should be measured on the search distribution or by play directly, not by prediction accuracy on sampled transitions.
Authors:Alex Kwon
Abstract:
Aligned language models refuse harmful requests, but a one‑line prefill ("Sure, here is") strips the refusal. We ask where and how it fails. The harm representation stays intact: on the prompts the attack flips to compliance, a linear probe reads harm as high as on the refused ones (0.91‑0.98), while behavioral refusal drops to chance. This holds across four models and three families (1.5‑3.8B, and at 14B). Refusal is therefore a shallow, response‑site computation. We localize it to an early window: a dose‑matched position control shows the first half of the response suffices to break refusal, while the second half is nearly inert. Three causal probes converge on that window. Restoring the harm direction there partially re‑engages refusal. Injecting the model's own refuse‑state reverses the jailbreak (74%, held‑out). And knocking out the early response's attention to the prefill, but not an equal attention mass elsewhere, selectively collapses the harmful continuation. A base‑model control identifies the mechanism: the same knockout collapses the continuation prefill‑specifically even in a non‑safety‑tuned base model (64% to 25% harmful content vs a matched control's 64%, replicated at 7B). So the prefill's grip is generic autoregressive conditioning, not safety‑specific suppression, and "refusal restoration" is a model‑dependent fallback. The dominant mechanism is passive. A small safety‑specific attractor remains on top (logit‑trace concentration 0.24 vs 0.03), whose active‑vs‑passive character we size but do not fully separate. No single direction or component is a clean handle either: the decision is decodable but distributed, and refusal tracks harm rather than scary surface. The consequence is structural: a monitor reading the untouched prompt‑side representation is immune by construction, but only to response‑site attacks. The mechanism is diffuse; the failure surface is local.
Authors:Christoph Kirsch
Abstract:
To answer a question about a program, move the program to where the question is decidable. Every such move is a translation, and every translation is a place to be wrong. We study translation as a graph ‑‑ many languages, a few reasoning targets, independently built routes of honestly different trustworthiness ‑‑ and give it a calculus: pairs of languages close commuting squares that are directional (exactness is the identity‑embedding special case of over‑approximation), checkable per program, and composable, a route's contract being the componentwise meet of its hops' contracts ‑‑ assurance class, direction, kept observables, measured cost. One asymmetry organizes trust: witness‑carrying answers are self‑certifying by replay at the source; universal answers are where grades, independent branches, and re‑checked certificates earn their cost. The compositional core, lax telescope included, is mechanized in Lean 4. hurdy‑gurdy implements the calculus as two planes meeting in one registry. The use plane reads declarations and produces evidence‑carrying answers; its builders and its intended player are both LLMs, untrusted by construction. The evolution plane grows the graph: unmet questions are recorded as demand, pairs are recommended by evidence and registered by humans, and a ratchet keeps every prior verdict standing. Answers never write; growth never answers. Run indefinitely, the loop converges on every reducibly decidable question, at fidelity that only rises. We measure the July 2026 snapshot ‑‑ per‑construct conjoined coverage, dual‑route branch agreement for two ISAs, source‑level witness replay, certified unreachability re‑validated by a formally verified checker, escape rates for the gate itself ‑‑ and report the defects the architecture caught in its own authors' work.
Authors:Ruijiang Dong, Zesheng Ye, Jianzhong Qi, Lei Feng, Feng Liu, Gang Niu, Masashi Sugiyama
Abstract:
Pre‑trained vision‑language models (VLMs) enable zero‑shot image classification by computing the similarity score between an image and textual descriptions, typically formed by inserting a class label (e.g., "cat") into a prompt (e.g., "a photo of a"). Since the score for a given image‑class pair is sensitive to the choice of prompt, existing studies ensemble multiple prompts using a weighting vector to aggregate scores across different prompts. Yet, in current strategies, the weighting vector assigned to each prompt is shared across all classes, implicitly assuming that prompts are conditionally independent of classes, which often does not hold in practice, as a prompt like "an aerial view of" might be apt for "airport" but ill‑suited for "apple". To address this, we propose class‑aware zero‑shot prompt reweighting (CARPRT). This scoring scheme adjusts the weighting vector for each class label by capturing the class‑specific relevance of different prompts in a training‑free manner. For each class label and every available prompt, we quantify their class‑specific relevance by averaging image‑text relevance scores over images predicted to that class under the given prompt. These estimates are then normalized to derive class‑specific weights. Evaluations on standard image classification benchmarks show that CARPRT outperforms existing class‑independent reweighting methods, confirming that modeling prompt‑class dependencies is crucial for effective zero‑shot prediction and even broader VLM‑based application settings that rely on prompt ensembling. Our code is available at https://github.com/tmlr‑group/CARPRT.
Authors:Yukun Song, Changwei Wang, Xingtian Pei, Shibiao Xu, Wenhao Xu, Shunpeng Chen, Yu Zhang, Ke Zhang, Rongtao Xu, Xuxiang Feng, Pengyang Wang
Abstract:
Inspired by how humans communicate spatial information, language‑guided geo‑localization has gained significant traction for its intuitive and practical value. Despite this progress, most methods still rely on a static, one‑shot retrieval paradigm, which fails to handle the ambiguity and incompleteness inherent in real‑world natural language descriptions. We propose a paradigm shift to reasoning retrieval and introduce Dialogue Place Recognition (DlgPR), which casts localization as an interactive, dialogue‑driven reasoning process. To support this new task, we present DlgQuest‑Cities, the first large‑scale dialogue‑based benchmark for place recognition, and a unified reasoning framework that couples a cross‑modal multi‑level retriever with an intelligent questioner, DQ‑pilot. DQ‑pilot is trained in a curriculum: supervised fine‑tuning on a curated DQ‑cities‑20k subset followed by reinforcement refinement on a harder DQ‑cities‑10k split via GRPO. Two task‑aligned metrics guide learning: a Discriminative Difficulty Index (DDI) for curriculum sampling and a Positional Retrieval Gain (PRG) reward that directly measures retrieval improvement induced by a question. Experiments show this reasoning‑based approach significantly outperforms baselines. The code and model are available at https://github.com/Graysonggg/DlgPR.
Authors:Ely Hahami, Ishaan Sinha, Lavik Jain
Abstract:
Can small language models detect and report on perturbations their own internal activations? We investigate this question through the lens of activation steering: injecting concept vectors into a model's residual stream and measuring whether the model can accurately report on the perturbation. We first show that the binary detection paradigm used in prior work ‑‑ prompting the model to answer Yes'' or No'' to whether it detects an injected thought ‑‑ is confounded in small models, as steering biases the model toward affirmative responses regardless of the question content. We therefore propose two confound‑free evaluation paradigms: sentence localization (identifying which of N sentences was perturbed, chance = 1/N) and strength comparison (identifying which of two sentences received a stronger injection, chance = 50%). Evaluating across six models from two families (Llama‑3.2 and Gemma‑4), we find that models as small as 2B parameters introspect reliably well above chance, and that introspective ability generally increases with scale. Llama‑1B, however, performs at or below chance. We then introduce \emphIntrospection Fine‑Tuning (IFT): supervised fine‑tuning on sentence‑localization examples constructed from the model's own perturbed forward passes. IFT raises Llama‑1B sentence‑localization accuracy from 9.6% to 60.6% (a 6× improvement), with gains generalizing zero‑shot to the held‑out strength‑comparison task (30.2% \to 52.2%). IFT also improves introspection for 3B and 8B models, while inducing negligible degradation on standard capability benchmarks. Our results suggest that introspective ability is not fixed by scale alone: it can be directly trained, and doing so unlocks latent self‑monitoring capacity with implications for AI transparency and alignment. Our code is \hrefhttps://anonymous.4open.science/r/IFT‑introspection‑2092/README.mdhere.
Authors:Zhihao Xie, Junfeng Wu, Xinting Hu, Junchao Huang, Li Jiang
Abstract:
Video generative models commonly rely on latent spaces learned by 3D Variational Autoencoders (3D‑VAEs). However, conventional 3D‑VAEs are mainly optimized for pixel‑level reconstruction, which can limit the semantic and spatio‑temporal structure captured by their latents. Meanwhile, Video Foundation Models (VFMs) such as V‑JEPA 2 and VideoMAEv2 show strong video understanding capabilities, yet whether their frozen representations can be transformed into compact, reconstruction‑capable, and generation‑friendly video latents remains largely unexplored. We answer this question with VideoRAE, a representation autoencoder that leverages multi‑scale hierarchical features from a frozen video foundation encoder and compresses them with a lightweight 1D self‑attention projector. VideoRAE supports both continuous latents for Diffusion Transformers and discrete tokens for autoregressive models via multi‑codebook high‑dimensional quantization. During decoding, a local‑and‑global representation alignment objective with the frozen VFM teacher improves semantic preservation and enables training without KL regularization. Experiments show that VideoRAE achieves strong reconstruction in both continuous and discrete regimes. On UCF‑101, it obtains state‑of‑the‑art class‑to‑video gFVDs of 40 and 93 with AR and DiT generators, respectively, while converging approximately 5x faster than competing autoencoder baselines. In a controlled 2B‑scale text‑to‑video study, replacing LTX‑VAE with VideoRAE leads to faster convergence under comparable settings. These results validate frozen VFM representations as versatile and generation‑friendly video latents. The model and code will be released on https://zhxie0117.github.io/VideoRAE.
Authors:Boyuan Wang, Zhenyuan Zhang, Zhiqin Yang, Peijun Gu, Shuya Wang, Xiaofeng Wang, Xianghui Ze, Yifan Chang, Guosheng Zhao, Jiangnan Shao, Guan Huang, Hengyu Liu, Yonggang Zhang, Wei Xue, Chunyuan Guan, Chenglin Pu, Yike Guo, Xingang Wang, Zheng Zhu
Abstract:
Autonomous data collection governs the volume and quality of real‑world trajectories for manipulation policy learning. Existing pipelines reduce human effort via self‑resetting, VLM verification, or language‑guided correction, yet episode‑scoped fixes must be reissued whenever the same failure recurs, so oversight cost grows with session length rather than with the number of distinct problems. We present Zero2Skill, a human‑robot symbiotic agentic system in which corrections are retained and reused across rounds. The collection loop collects, verifies, and resets autonomously, pausing for a remote operator only when a phase exhausts an explicit retry budget. An LLM parser maps each natural‑language utterance to a structured adjustment stored in Corrective Memory, so addressed failure modes typically need not be corrected again under the same conditions. On a real‑robot desktop‑clearing testbed, Zero2Skill matches teleoperation episode success while reducing human working time to 16%. Language corrections improve verifier‑human agreement in all four evaluated settings and raise average single‑attempt success from 12.5% to 47.5% (arm‑selection: 20.0% to 50.0%). Policies fine‑tuned on Zero2Skill data match teleoperation‑trained policy success at a fraction of collection human cost.
Authors:Katie Everett
Abstract:
We investigate how each component of the Transformer feedforward block architecture design determines how much rank survives across depth at initialization. We reinterpret skip connections and normalization, long understood as controlling magnitude, as mechanisms for preserving gradient rank across depth, since the very matrix multiplications and nonlinear activations that make the network expressive also reduce the rank. We show that skip connections trade off rank collapse against ensemble‑like behavior, controlled by the relative scales of the branch and the skip: skip connections route the gradient around the residual branch, where rank is lost, rather than along the long gradient paths that encourage the layers to compose. The placement of the normalization layer controls this same tradeoff by setting the branch‑to‑skip ratio across depth, unifying much of the normalization placement and depth scaling literature, in particular why rank collapses for Post‑Norm but plateaus for Pre‑Norm. Other aspects of the architecture, like the two‑matrix structure that expands and contracts the width, use additional parameters to preserve the representation or branch Jacobian rank. The second matrix decorrelates a coherent mean spike that would grow across blocks with a single matrix and uncentered activation, preventing the residual representation from collapsing. The width expansion between the two matrices keeps the branch Jacobian full rank: applying the rank‑reducing activation in this expanded space leaves enough directions to span the original, at a width that follows a Marchenko‑‑Pastur law. The initialization rank of the input‑‑output Jacobian predicts which networks train on CIFAR‑10. Taken together, we recast architecture design for deep networks as navigating an intrinsic tradeoff among rank collapse, ensemble‑like behavior, and parameter count.
Authors:GigaWorld Team, Angen Ye, Angyuan Ma, Boyuan Wang, Chaojun Ni, Fangzheng Ye, Guan Huang, Guo Li, Guosheng Zhao, Haodong Yan, Hengtao Li, Jiwen Lu, Kai Wang, Mingming Yu, Qitang Hu, Qiuping Deng, Songling Liu, Xiaoyu Tian, Xiaofeng Wang, Xinyu Zhou, Xiuwei Xu, Xinze Chen, Yang Wang, Yejun Zeng, Yifan Chang, Yun Ye, Zhenyu Wu, Zhanqian Wu, Zheng Zhu
Abstract:
World Action Models (WAMs) improve robot policy learning by jointly modeling actions and future visual observations, using future scene evolution as dense supervision for physically grounded action generation. However, a common design in existing WAMs is to explicitly generate future videos at inference time, incurring substantial computational overhead and hindering real‑time closed‑loop deployment. GigaWorld‑Policy addresses this issue with an action‑centered formulation, where future visual dynamics are used during training while action‑only decoding is used at inference time. Building upon this framework, we present GigaWorld‑Policy‑0.5, an enhanced action‑centered WAM designed for more efficient robot control. During pretraining, GigaWorld‑Policy‑0.5 adopts a mixed Action‑Conditioned World Modeling (AC‑WM) and WAM training strategy. This strengthens the coupling between visual dynamics and robot actions and improves the transferability of action representations for downstream policy learning. For efficient inference, GigaWorld‑Policy‑0.5 introduces a Mixture‑of‑Transformers architecture that separates visual dynamics modeling and action generation into specialized experts, reducing active computation during action‑only inference and achieving 85 ms inference latency on a local RTX 4090 setup. In addition, we employ an agent‑based AutoResearch pipeline to systematically search training configurations, enabling more efficient identification of optimal experimental setups while reducing the time and manual intervention required for hyperparameter tuning. Experiments and ablations show that GigaWorld‑Policy‑0.5 preserves the training benefits of future visual dynamics while improving inference efficiency for robot control.
Authors:Geng Li, Haiwen Li, Rui Chen, Jing Tang, Lei Sun, Xiangxiang Chu
Abstract:
Video aesthetic assessment (VAA) aims to predict how aesthetically pleasing a video is, yet remains far less explored than other visual assessment tasks. Its progress is hindered not only by the scarcity of large‑scale benchmarks, but also by the intrinsic subjectivity of aesthetic judgment, which is shaped by human perception. In this paper, we revisit VAA from a psychological perspective and propose Peak‑End‑Net, a lightweight and interpretable framework inspired by the peak‑end rule, which suggests that people tend to judge a temporal experience mainly according to its salient moments and the ending. Building on this intuition, we first transfer knowledge from image aesthetic assessment (IAA) to VAA by introducing a pretrained IAA head to produce frame‑wise aesthetic priors, which serve as surrogate signals for identifying aesthetically salient moments and guiding peak‑end rule‑based temporal aggregation. To further capture how a video evolves aesthetically over time, we design an aesthetic rhythm encoder that models temporal progression beyond isolated moments. Additionally, we refine the overall assessment through a dynamic gated fusion mechanism to improve robustness under distribution shift. Our method is built on a frozen vision transformer (ViT) and requires only a small number of trainable parameters, making it scalable and parameter‑efficient. Extensive experiments on two existing VAA benchmarks, including in‑domain evaluation on VADB and cross‑domain testing on DIVIDE‑3K, demonstrate that our approach achieves state‑of‑the‑art performance, affirming the value of psychologically grounded modeling for VAA. Our code and models are available at https://github.com/AMAP‑ML/Peak‑End‑Net.
Authors:Haoran Li, Jiebi Deng, Tong Jin, Jinghong Han, Yuxin Wang, Zexin Wang, Qingyi Si, Weikang Gong, Xiahai Zhuang, Jia You, Wei Cheng, Jianfeng Feng, Hongcheng Guo
Abstract:
Personal health management unfolds over repeated encounters, yet most health AI systems treat each request in isolation. We developed HealthClaw, an open‑source agent architecture that updates support as a person's routines, preferences, measurements and risks change. It separates shared safety rules and medical knowledge from private longitudinal memory containing profile facts, reusable procedures and episodic traces. After each episode, induction determines what should update the profile, revise a procedure, remain episodic or be excluded. We evaluated HealthClaw with a synthetic year‑long benchmark and nine 200‑case biomedical tasks. Across 900 longitudinal support probes, answer accuracy increased from 0.2% with current‑query prompting to 45.7% with HealthClaw, while prompt‑side context exposure was 71.7% lower than with full‑history prompting. In 100 privacy probes, HealthClaw produced higher privacy‑aware answer quality and fewer unsafe disclosures than both baselines. Across the biomedical tasks, the mean absolute gain in the task‑specific primary metric was 27.0 percentage points, and seven gains remained significant after false‑discovery‑rate correction. These offline benchmarks support governed, self‑evolving memory for longitudinal personal health agents, although clinical effectiveness requires prospective evaluation. HealthClaw is publicly available at https://github.com/HC‑Guo/HealthClaw.
Authors:Thang-Anh-Quan Nguyen, Moussab Bennehar, Luis Guillermo Roldao Jimenez, Nathan Piasco, Dzmitry Tsishkou, Laurent Caraffa, Jean-Philippe Tarel, Roland Brémond
Abstract:
Reliable perception under diverse weather conditions remains a major challenge for autonomous driving systems. A common strategy to improve robustness is either to synthesize adverse weather conditions for training perception models or to apply weather‑removal techniques to recover clean inputs. However, existing approaches typically rely on synthetic data augmentation or physics‑based, task‑specific models that require paired training data and often struggle to generate realistic weather effects or generalize robustly to out‑of‑domain scenarios. Toward this problem, we present Cyclone, a unified framework for weather editing based on latent diffusion, equipped with cycle‑consistent constraints and knowledge from image‑text models. Cyclone enables the generation of multiple weather conditions across diverse scenes while eliminating the need for paired data. Experimental results show that our approach produces more realistic, structure‑preserving outputs than existing baselines and leads to consistent improvements across several downstream driving perception tasks. Furthermore, we demonstrate that Cyclone can be distilled to a video diffusion model for temporally consistent weather editing.
Authors:Jiangang Han
Abstract:
Serial verification gates are a core reliability primitive in LLM harnesses: a candidate answer is returned only if k verifier calls all accept it. Under conditionally independent gates, the recent Odds Law (arXiv:2606.15712) shows that posterior log‑odds grow linearly in k, so failure decays exponentially, and states that "a tight theory of partially correlated verifier cascades remains open." This note gives a minimal such theory. Modeling the per‑instance false‑accept rate on the generator's own errors as a latent variable α~ G (de Finetti), the exact cascade posterior is \ell_k = \ell_0 ‑ \ln m_k, with m_k the k‑th moment of G. Then: (i) \ell_k is concave in k for every non‑degenerate G ‑‑ the Odds Law is its tangent at the first gate and an upper bound; (ii) for Beta(a,b) latents, failure decays polynomially, 1‑r_k \asymp k^‑b, with correlation parameter ρ_v = 1/(a+b+1); (iii) a blind‑spot atom of mass 1‑π at α=1 caps the evidence extractable from any number of gates at ‑\ln(1‑π) nats, so reliability saturates below 1; (iv) letting the true‑accept rate also vary (β~ H) yields a trichotomy ‑‑ gates eventually always help, plateau, or actively harm ‑‑ decided by the upper‑tail exponents of G and H, with closed‑form crossover k^\dagger. The mechanism is survivorship: errors surviving gates are the high‑α ones. The theory is measurable: R repeated verdicts per instance identify the first R moments of G, so two verdicts identify ρ_v; beta‑binomial likelihood and NPMLE recover the reliability curve and the ill‑posed ceiling. In synthetic tests, independence‑based extrapolation underestimates failure by 20x at k=5 and ~3000x at k=10; the correlated fit at R=8 tracks held‑out depths. The practical lever is decorrelation ‑‑ changing model family, modality, or evidence source ‑‑ not adding gates.
Authors:Yexin Hu, Haoyi Zheng, Johannes Heidersberger, Dongheui Lee
Abstract:
Learning from demonstration (LfD) enables robots to learn manipulation skills directly from expert demonstrations but remains challenging for contact‑rich tasks involving geometric constraints and force interaction. Existing approaches typically require multiple complete demonstrations and do not support reverse skill execution. In this paper, we present a unified one‑shot framework for constrained manipulation that learns both forward and reverse execution from a single, possibly unfinished demonstration. Our method decomposes demonstrations into non‑contact and contact phases, with non‑contact motion encoded with dynamic movement primitives (DMP), and contact motion represented as a sequence of screw motion primitives segmented by our proposed geometry‑driven twist‑direction segmentation algorithm. During execution, screw primitives are executed sequentially under admittance‑guided pose correction and speed regulation, enabling task completion beyond the demonstrated trajectory length as well as reverse skill execution without additional learning data. Experiments on peg insertion, battery insertion, lock opening, and screw driving tasks demonstrate improved success rates and robustness over segmentation and one‑shot trajectory learning baselines. Details are available on the project website: https://tuwien‑asl.github.io/LfD‑Screw/.
Authors:Zhuoyuan Fu, Zeshang Li, Yiqiong Zhang, Hangui Lin, Yan Shu, Yan Li, Binyang Li, Yaru Zhao
Abstract:
While Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in 2D medical image understanding, their extension to 3D volumetric imaging remains hindered by prohibitive annotation costs and dataset opacity. Current data formats, predominantly consisting of rigid Visual Question Answering (VQA) pairs or unstructured final clinical reports, typically fail to capture explicit clinical reasoning. To address this limitation, we introduce a large‑scale structured reasoning dataset constructed via a novel slice‑wise data synthesis paradigm. Inspired by the genuine diagnostic workflow of radiologists, this paradigm models visual cognition by decomposing the complex 3D reading process, translating global clinical priors into fine‑grained, per‑slice observations that are subsequently synthesized into an interpretable Chain‑of‑Thought (CoT). Crucially, this synthesized reasoning framework enforces essential clinical principles: sequential spatial tracking, multi‑slice spatial awareness for artifact mitigation, and differential exclusion. To validate this approach, we instruction‑tune a standard 2D‑pretrained MLLM baseline using the synthesized data to enhance its volumetric comprehension. Comprehensive evaluations across multiple 3D medical benchmarks demonstrate that our method yields significant performance improvements over the 2D baseline. Furthermore, the resulting model exhibits robust spatial reasoning capabilities and rivals resource‑intensive native 3D architectures, effectively bridging the performance gap. Ultimately, this data‑centric strategy unlocks deep volumetric understanding and highly interpretable clinical logic without requiring computationally expensive 3D‑specific pre‑training. The complete repository, including datasets and training workflows, is publicly available at https://github.com/2020420145009/hounsfield.
Authors:Zhenkai Zhang, Krista A. Ehinger, Tom Drummond
Abstract:
We introduce TCAM‑Diff, a novel 3D medical image generation model that reduces the memory requirements to encode and generate high‑resolution 3D data. This model utilizes a decoder‑only autoencoder method to learn triplane representation from dense volume and leverages generalization operations to prevent overfitting. Subsequently, it uses a triplane‑aware cross‑attention diffusion model to learn and integrate these features effectively. Furthermore, the features generated by the diffusion model can be rapidly transformed into 3D volumes using a pre‑trained decoder module. Our experiments on three different scales of medical datasets, BrainTumour 128 x 128 x 128, Pancreas 256 x 256 x 256, and Colon 512 x 512 x 512, demonstrate outstanding results. We utilized MSE and SSIM to assess reconstruction quality and leveraged the Wasserstein Generative Adversarial Network (W‑GAN) critic to assess generative quality. Comparisons with existing approaches show that our method gives better reconstruction and generation results than other encoder‑decoder methods with similar‑sized latent spaces.
Authors:Neel Kelkar, Simon Niedermayr, Kaloian Petkov, Klaus Engel, Rüdiger Westermann
Abstract:
Recent extensions of 3D Gaussian Splatting (3DGS) capture fine color details using hash‑grid‑based appearance parameterization but incur high computational cost during fragment rendering. We introduce a decoupled radiance representation that models low‑frequency geometry and view dependent appearance features with 2D surfels while representing high‑frequency textures via a view‑independent spatial hash grid that is baked into a compact texture atlas. By including sparsity‑enhancing optimizations that penalize semi‑transparency and per‑primitive falloff, our method aggressively prunes insignificant surfels and achieves significantly faster and sparser reconstructions than prior work. Exploiting geometric sparsity and efficient GPU texture mapping, our approach achieves up to a fivefold speedup over 3DGS while preserving state‑of‑the‑art visual fidelity, enabling real‑time 4K rendering at 60 FPS on consumer hardware.
Authors:Kui Jiang, Runzhe Li, Zhaocheng Yu, Guanglu Sun, Junjun Jiang, Xianming Liu
Abstract:
Video deraining aims to recover clean visual content from rainy videos for reliable perception under adverse weather. Existing methods mainly rely on RGB sequences and temporal redundancy, but RGB‑only restoration remains ambiguous in dynamic rainy scenes, where rain streaks, textures, boundaries, motion, and occlusions may share similar visual patterns. Event cameras provide complementary motion‑sensitive cues with high temporal resolution, but event streams also contain sensor noise and background‑triggered responses, so direct RGB‑Event fusion may introduce cross‑modal interference. To address this issue, we propose RainDancer, a progressive RGB‑Event video deraining framework based on a decompose‑before‑interact paradigm. The core idea is to separate rain and background components within each modality before cross‑modal interaction. In the RGB branch, frame features are progressively decomposed into rain and background representations. In the event branch, a rain‑oriented spiking neural network module captures sparse and bursty event dynamics associated with rain motion. Component‑level fusion is then performed between semantically aligned representations for structure preservation and rain suppression. We further introduce event‑domain supervision to regularize sparse event reconstruction, structural consistency, and gradient orientation. Experiments on synthetic and real RGB‑Event video deraining datasets demonstrate superior quantitative performance, visual quality, and downstream perception robustness. Code is available at https://github.com/AE86‑plus/RainDancer.
Authors:Junlong Li, Junxi Li, Yuxiang Yang, Wenbin Zou, Lap-Pui Chau, Yi Wang
Abstract:
Most daily activities are inherently procedural. However, existing evaluations for egocentric video understanding seldom address procedural understanding and largely overlook complex key‑step‑level reasoning under the widely used video question answering (VQA) paradigm for MLLMs. Such capabilities are crucial for building procedural AI assistants deployable on wearable devices. To bridge this gap, we introduce the Egocentric Procedural Understanding VQA task (EgoProceVQA), which systematically evaluates egocentric procedural reasoning abilities of current MLLMs and agents through six types of key‑step‑centric questions. Furthermore, we develop EgoProceGen, a data generation platform that efficiently constructs QA data tailored to different question types. Based on this platform, we build a benchmark with 3,600 questions, four common procedural scenarios, and 31 everyday procedural tasks. Evaluations on EgoProceVQA show that existing MLLMs and agents still have substantial room for improvement in procedural understanding. Therefore, we further propose EgoProceAgent, a self‑skill‑exploration agentic framework. We design a generic tool library for procedural understanding and a standardized sub‑skill library shared across tools and models, enabling self‑exploration without ground‑truth supervision. By exploring how to compose and select sub‑skills, the agent discovers effective skill strategies for diverse problems, and attains state‑of‑the‑art performance among open‑source models on multiple tasks. Together, our benchmark, generation platform, and agentic framework establish a unified foundation for EgoProceVQA. Project page: https://z1oong.github.io/EgoProceVQA/.
Authors:Dikshit Chauhan
Abstract:
Multimodal optimization aims to locate multiple globally optimal or near‑optimal solutions in a single run. This paper presents \emphS‑CARD‑CMSA, a score‑aware candidate‑archive and density‑filtered reporting framework built on the covariance matrix self‑adaptation evolution strategy with repelling subpopulations (RS‑CMSA‑ESII). The method is developed for the IEEE CEC 2026 Competition on Benchmarking Niching Methods for Multimodal Optimization. Rather than modifying the core search dynamics of RS‑CMSA‑ESII, S‑CARD‑CMSA preserves its sampling, covariance adaptation, taboo‑region update, restart, and termination mechanisms. Two conservative extensions are introduced. First, a passive secondary candidate archive records the restart‑level best candidates without influencing the search trajectory. Second, a score‑aware density‑filtered reporting rule constructs the final solution set by balancing robust peak ratio and precision‑driven F1‑score. Development experiments show that the density‑filtered rule preserves the peak coverage obtained by a medium score‑aware rule while reducing redundant reports. On a broader validation subset, it maintains the same mean RPR while improving mean precision, F1‑score, and the official‑score‑oriented average. The method does not use true global‑minimum locations during optimization; such information is used only for offline development analysis and post‑run scoring. The source code of S‑CARD‑CMSA is available at https://github.com/ChauhanDikshit.
Authors:Shuhao Li, Guodong Du, Anhao Zhao, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen
Abstract:
Large language models have made strong reasoning gains through supervised fine‑tuning, reinforcement learning, and on‑policy distillation, yet these post‑training methods are usually evaluated only by final‑answer accuracy. We study how they reshape confidence during reasoning. We introduce a three‑stage calibration framework that evaluates confidence before, during, and after chain‑of‑thought generation, corresponding to difficulty estimation, early termination, and answer aggregation. Through a controlled comparison on mathematical reasoning benchmarks, we find that OPD provides the most useful pre‑reasoning confidence, SFT gives the strongest online signal for early stopping, and RL produces the most reliable trace‑level signal for aggregation. We further show that confidence reliability is position‑dependent: RL confidence becomes informative after a path‑commitment phase, while OPD confidence is useful early but can become inversely calibrated later. Based on this observation, we propose PosConf, a position‑aware confidence strategy that uses confidence only from reliable relative‑position intervals. PosConf improves RL answer aggregation by 6.1 points over majority voting and consistently improves OPD early stopping under tight token budgets, with gains up to 4.3 points by avoiding its later inverse‑calibration region, showing that \emphconfidence in reasoning models should be used both stage‑wise and position‑awarely. Our code is available at https://github.com/EIT‑NLP/Post‑Training‑Calibration.
Authors:Ashish Thapa, Samrat Karki
Abstract:
We built a compact convolutional network (1.11 M parameters) for 46‑class DHCD Devanagari recognition and reached 99.73%, the highest reported at 15.6x smaller than prior state‑of‑the‑art. We have effectively reached the saturation point: every model tested, large teacher ensembles included, hits the same 11‑error intrinsic floor. No configuration achieves a statistically clear win under exact McNemar tests with Wilson confidence intervals. Even without knowledge distillation, our student matches the nearest large‑model baseline (17.32 M parameters; McNemar p = 0.345). Outside of DHCD, zero‑shot on CMATERdb digits gives 76.6% and fine‑tuning reaches 97.8%; corruption robustness is also far better than large baselines (mean corruption accuracy 75.7% vs. 38.7%). All artifacts are at https://github.com/Ampixa/barnamala.
Authors:Zehan Liu, Yage He, Xianwu Gong
Abstract:
Existing iterative stereo matching methods primarily adopt two types of correspondence representation: explicit matching search via correlation volumes and local residual refinement via warped features, yet the two remain separately modeled. We propose WAVE‑Stereo, built on a core insight: correlation volumes and feature warping provide complementary matching cues. GeoWarp Correspondence Encoder (GWCE) encodes matching search, residual alignment, and disparity prior in parallel at the ConvGRU input. To mitigate matching degradation in textureless regions, we propose Periodic Global Context Propagation (PGCP), which propagates global spatial information in a periodic manner. On five real‑world benchmarks ‑‑ Middlebury, ETH3D, KITTI 2012, KITTI 2015, and Booster ‑‑ WAVE‑Stereo achieves competitive zero‑shot generalization accuracy without any external foundation model prior, achieving 3.18% D1‑all on KITTI 2015, 4.42% Bad‑2.0 on Booster, and 66ms real‑time inference, striking a favorable balance between accuracy and efficiency. Our code is available at https://github.com/yamanoko‑do/WAVE‑Stereo.
Authors:Zijie Yu, Gaowen Liu, Ramana Rao Kompella, Philip S. Yu, Yue Song
Abstract:
Contrastive Language‑Image Pretraining (CLIP) representations form a semantic embedding space governed by cosine similarity, reflecting an intrinsic hyperspherical geometry. However, existing probabilistic interpretations typically rely on Gaussian assumptions, which fail to capture this directional and multimodal structure. We propose a principled density model for the CLIP latent space based on Mixtures of von Mises‑Fisher (MovMF) distributions defined on the unit hypersphere. Using the Expectation‑Maximization (EM) algorithm, we efficiently learn a probabilistic model in which each mixture component corresponds to a coherent semantic concept. This formulation yields a closed‑form likelihood naturally aligned with hyperspherical geometry, enabling accurate and interpretable density estimation. Empirically, our model significantly improves long‑tailed and out‑of‑distribution detection and provides a natural semantic decomposition, representing each embedding as a sparse probabilistic combination of interpretable concepts. These results suggest that CLIP latent space is more faithfully characterized as a hyperspherical semantic mixture rather than an isotropic Gaussian, establishing a simple and geometrically consistent probabilistic framework for modeling and understanding multimodal representations. Project page is available at https://xiaoyuzhizi.github.io/movmf‑clip/.
Authors:Boyu Mi, Mengchen Ma, Yifei Yao, Xing Gao, Junting Chen, Yangzi Li, Zihou Zhu, Guohao Li, Zhenfei Yin, Tai Wang, Yao Mu, Jiangmiao Pang, Hanqing Wang
Abstract:
Real‑world deployment of embodied agents requires active exploration, visual grounding, and interactive intent disambiguation. However, existing frameworks often rely on privileged simulator states or assume complete instructions, bypassing realistic deployment challenges. To bridge this gap, we present REAL, an agentic framework for open‑world mobile manipulation. REAL establishes sim‑to‑real‑consistent environment APIs without oracle perception and integrates a simulated user to enable human‑in‑the‑loop interaction. Within this environment, we design diverse task compositions to drive data collection, supervised fine‑tuning, and online reinforcement learning, systematically optimizing agent performance. To comprehensively evaluate this approach, we introduce REAL‑Bench, a benchmark spanning 241 tasks across active exploration, visual distraction, articulated manipulation, and interactive disambiguation. Experimental results demonstrate that our trained agent outperforms leading commercial closed‑source VLMs on interactive tasks with a 56.9% success rate. Further empirical analysis reveals that our hierarchical training pipeline successfully aligns the model's tool‑use capabilities while maintaining robust open‑vocabulary reasoning under extended exploration horizons. Finally, we deploy and evaluate our framework on a physical dual‑arm mobile robot, where it achieves a 78.3% end‑to‑end success rate over 60 real‑world episodes. These physical trials demonstrate robust zero‑shot transferability to unseen household scenarios, validating that our sim‑to‑real‑consistent design successfully bridges the reality gap for long‑horizon mobile manipulation. Code is available at https://github.com/InternRobotics/REAL.
Authors:Yuan Xu, Youheng Shi, Chengyang Li, Wentao Zhu, Yizhou Wang
Abstract:
Vision‑Language‑Action (VLA) models inherit rich semantic representations from pretrained Vision‑Language Models, yet fine‑tuning on limited robot demonstrations degrades this structure and undermines generalization. A fundamental question therefore arises: what constitutes a good action representation? Inspired by the mirror neuron theory's insight that observation and execution share an intention‑level encoding, we examine whether a robot's action representations preserve the semantic structure captured by pretrained encoders. Systematic probing confirms that this structure erodes during finetuning, and that its quality synchronizes with both task success and out‑of‑distribution generalization. We further introduce a plug‑and‑play method that anchors action representations to a semantic manifold while decomposing representations into a shared semantic channel and a private channel, all discarded at inference, leaving the deployed model unchanged. Validated on different VLA backbones across simulation and real‑world benchmarks, our method yields up to +18.7% on real‑world in‑distribution tasks and +21.5% on out‑of‑distribution generalization.
Authors:Xian Li, Rong Wei, Lujie Yang, Haolin Huang, Junyuan Fang, Siliang Tang, Jun Xiao, Rui Tang, Juncheng Li
Abstract:
Physically grounded 3D assets are increasingly important for embodied AI and robotic simulation. However, most existing 3D assets lack unified physical semantics, including articulation semantics and intrinsic physical properties, required for realistic interaction. Current approaches either treat these semantics independently or rely on canonicalized object structures, limiting robustness across heterogeneous 3D assets. We present UniPhys, a scalable framework for automatically transforming raw 3D assets into simulation‑ready assets with unified physical semantics. Based on UniPhys, we construct UniPhys‑40K, a large‑scale physically grounded dataset, together with UniPhys‑Bench, a carefully verified benchmark for unified physical grounding evaluation. We further introduce UniPhysGen, a unified physical grounding model that jointly reasons over articulation semantics and intrinsic physical properties. UniPhysGen incorporates geometry‑robust articulation grounding to mitigate geometric shortcut bias under heterogeneous part decompositions. Extensive experiments demonstrate state‑of‑the‑art performance across articulation grounding and intrinsic physical property estimation tasks, while the resulting assets can be directly deployed in robotic simulation environments for realistic physical interaction. Our code and dataset will be available at https://github.com/breezexian/UniPhysGen.
Authors:Grzegorz Brzezinka
Abstract:
Can a language model estimate its familiarity with an entity before generating an answer? We study activations at the final prompt token in twelve instruction‑tuned models from the Bielik, PLLuM, Gemma‑4, and Qwen3 families, using a new dataset of 1,440 Polish entities spanning four domains and ten Wikipedia‑pageview deciles, plus fabricated controls. Familiarity‑probe scores separate real from fabricated entities in every family; in the Polish‑adapted Bielik and PLLuM families they additionally track entity popularity (model‑mean Spearman ρ 0.28‑0.57, versus at most 0.11 in Gemma‑4 and Qwen3), a pattern more strongly associated with Polish adaptation than with parameter count in this model sample. In a paired experiment on two families, probes retain 96‑101% of within‑language AUROC when the Polish question stem is replaced with an English one around unchanged entity names, showing robustness to prompt language in this setting. In Gemma‑4‑12B, the only model that natively refuses, adding a one‑dimensional familiarity direction at a single layer moves refusal rates monotonically in both directions (0.24 to 1.00 on well‑known entities; 0.73 to 0.00 on unknown ones). Finally, a calibrated familiarity probe is competitive among pre‑generation abstention gates, although post‑generation detectors better predict behavioral error on average. These results support a graded pre‑generation entity‑familiarity readout, and a separation between representational familiarity and the policy that converts it into abstention.
Authors:Songyu Xu, Xin Wang, Qiang Chen, Xinran Wang, Muxi Diao, Yuxuan Zhang, Kongming Liang, Rui Lin, Zhanyu Ma
Abstract:
Recent video generation models (VGMs) have made substantial progress in visual fidelity, yet their ability to follow long, compositional instructions remains insufficiently evaluated. Existing evaluation protocols often rely on prompts that are short and semantically shallow, with limited atomic constraints and weak spatio‑temporal dependencies. They also frequently depend on costly human evaluation or handcrafted vision pipelines, while providing little diagnostic insight into which instruction constraints succeed or fail. To address this gap, we propose VGIF‑Score, a highly automated and interpretable framework for evaluating instruction following in video generation. VGIF‑Score consists of two complementary components: an objective completion branch that parses prompts into a Spatio‑Temporal Directed Acyclic Graph (ST‑DAG) and performs dependency‑aware QA with short‑circuit diagnostics, and a subjective satisfaction branch that uses instruction‑conditioned AutoRubric to assess cinematography, visual purity, motion smoothness, and physics adherence. Together, these components produce a unified score that captures both objective completion and perceptual satisfaction. We instantiate this framework on VGIF‑Bench, a benchmark of 223 long, structurally entangled prompts paired with approximately 4.3K fine‑grained evaluation items. Experiments on 14 proprietary and open‑source VGMs across more than 3K generated videos show that VGIF‑Score provides reliable, interpretable, and diagnostically useful evaluation of video generation instruction following. The code will be available at https://github.com/PRIS‑CV/VGIF‑SCORE.
Authors:Changqing Zhou, Yueru Luo, Yulan Guo, Bing Wang, Jie Qin, Changhao Chen
Abstract:
Accurate 3D scene understanding is fundamental to embodied intelligence and autonomous driving, where 3D occupancy provides a unified representation of objects, structures, and free space. However, recovering such a complete volumetric representation from visual observations remains challenging, particularly in occluded and unobserved regions. Visual geometry priors offer strong and generalizable geometric cues for addressing this challenge, but their outputs are inherently surface‑centric, whereas occupancy prediction requires reasoning about volumetric interiors and free space. To bridge this gap, we introduce GPOcc, which transforms visual geometry priors into occupancy‑aware sparse Gaussian representations for efficient and expressive volumetric scene modeling. Building on GPOcc, GPOcc++ models multi‑view observations and temporal sequences within a unified framework, allowing spatial and temporal evidence to be handled through the same representation. We further extend GPOcc++ from indoor scenes to outdoor occupancy prediction. Extensive experiments on both indoor and outdoor benchmarks demonstrate consistently strong performance across both multi‑view and temporal settings, together with favorable efficiency and generalization. Code will be released at https://github.com/JuIvyy/GPOcc.
Authors:Philip Huang, Chenrui Gao, Jiaoyang Li
Abstract:
Multi‑robot‑arm motion planning is a key challenge in deploying multiple manipulators for industrial tasks such as manufacturing. Existing search‑based and sampling‑based solvers often require significant computation time to produce collision‑free, high‑quality motions suitable for safe real‑world execution. In this work, we introduce a new suite of multi‑robot‑arm motion planners capable of near real‑time motion generation, combining classical planning algorithms with state‑of‑the‑art vectorized collision‑checking techniques. Based on CPU SIMD instructions, our new planners accelerate their primary bottleneck, collision checking, and achieve up to two orders of magnitude speedup in both motion planning and execution postprocessing for multi‑arm manipulation tasks. We also release our implementation to lower the barrier for research and development of multi‑robot‑arm planning and manipulation problems. Code is available at https://vamp‑mr.github.io/vamp‑mr
Authors:Kai Hsu Tsai, Yong Wei Fu, Hung I Yang, Yu-Chih Chen
Abstract:
Music visualization offers a powerful way to enhance listeners' understanding and experience of music by translating auditory signals into visual forms. However, most existing approaches either rely heavily on lyrics or generate flat, non‑immersive videos similar to conventional music videos, which limits their ability to convey the emotional dynamics of music and provide an immersive listening experience. We propose Bring Music The Horizon, an emotion‑aware pipeline for music‑driven 360^\circ video generation. Given an input song, our work first estimates its emotional trajectory by predicting valence‑arousal values at the level of every four bars. These values are then converted into emotion‑aware visual guidance using EmotiCrafter, and these guidance vectors can be manipulated by the SEGA framework, which provides fine‑grained semantic control for keyframe generation. Finally, image‑to‑video models are applied to the generated keyframes to synthesize temporally continuous 360^\circ videos for immersive music visualization. Our pipeline generates 360^\circ music visualization videos that reflect the emotional progression and temporal structure of the input song. We demonstrate its capability using songs from different genres and provide qualitative comparisons with From‑Sound‑To‑Sight, a representative audio‑to‑visual generation baseline, on our project page at https://etoile‑et‑toi‑mp3.github.io/BMTH_Project_Page/.
Authors:Bin Zang, Wenting Zheng, Xiaoliang Luo, Zhiyuan Fang, Shi Li, Lvchun Wang, Wei Yu, Yi Zhao, Tian Xie, Yuchi Huo, Rengan Xie
Abstract:
Recently, a line of works can generate impressive 3D objects from a single image, but they are limited by restricted representation resolution, making them unsuitable for 3D scene generation. In this work, we introduce HIVE‑3D, a novel method for high‑quality 3D scene generation based on hierarchical voxel enhancement framework. Specifically, given a single scene image as input, we first produce a coarse initial scene, then introduce image segmentation and attention‑based retrieval to align 2D image components with 3D scene components. Subsequently, we organize these scene relations into a hierarchical component tree, where nodes closer to the leaves denote finer‑grained components. Finally, we propose a voxel super‑resolution model that generates refined voxels for the target instance while maintaining strong consistency with the coarse voxels. Equipped with this model, we perform coarse‑to‑fine hierarchical super‑resolution on images and voxels for each component, producing a high‑resolution and high‑quality 3D scene. Extensive experiments demonstrate that our method significantly outperforms previous approaches, achieving state‑of‑the‑art performance.
Authors:Huatao Li, Xinwei Geng, Yuheng Wang, Yutong Li, Runde Yang, Hantao Chen, Shu Yao, Jingru Fan, Xuhui Ren, Yuanyuan Zhao, Fei Huang, Chen Qian
Abstract:
LLM‑based agents have rapidly improved at operating individual digital environments such as mobile applications, desktop systems, and smart homes. However, real‑world user goals often span multiple devices: information may come from a phone, be processed on a desktop, and the result may need to appear on another device. Most existing benchmarks center on a single dominant execution environment, making it difficult to evaluate whether agents can acquire and integrate information across heterogeneous devices and complete end‑to‑end tasks with cross‑device dependencies. We introduce DevicesWorld, a large‑scale executable benchmark for cross‑device collaborative operation. DevicesWorld contains 6,140 tasks and integrates three classes of device environments ‑‑ mobile, desktop, and IoT ‑‑ into a unified cross‑device interaction and evaluation framework. Each task defines a natural‑language user goal, participating devices and initial states, executable actions, rule‑based verifiers, and a cleanup procedure. A multi‑stage construction and quality‑control pipeline keeps tasks close to realistic user needs while allowing final outcomes to be automatically verified from device states and generated files. We evaluate five frontier LLM‑agent systems on a fixed evaluation set. All methods achieve low success rates, with the best reaching only 12.5%. Among failed runs, about 28.7% satisfy at least one scoring condition yet still fail the full task. Trajectories show that agents become stuck acquiring information or manipulating interfaces, confuse source and output devices, or terminate before all conditions are jointly satisfied. DevicesWorld turns cross‑device collaborative operation into an executable, reproducible, and diagnostically useful evaluation problem for research on reliable cross‑device agents.
Authors:Qingrong He, Lin Zhao, Kevin Zheng, Liang Lin
Abstract:
Vision‑and‑Language Navigation (VLN) necessitates an embodied agent to navigate in the physical world by adhering to natural language instructions. Recent advancements in Vision‑Language Models (VLM) have propelled the development of VLM‑based VLN methods with two predominant paradigms: (1) imitation learning (IL) on expert demonstrations, followed by the Dataset Aggregation (DAgger) algorithm to bolster error recovery capabilities; (2) reinforcement learning (RL) driven by verifiable rewards to enhance reasoning and exploration. A notable gap is the absence of integration between these two distinct paradigms. This paper introduces JOP‑VLN, a novel VLN framework that synergistically combines off‑policy imitation learning and on‑policy exploration within a three‑stage training pipeline. Initially, IL is employed on expert demonstrations to acquire basic navigation skills. Subsequently, the DAgger algorithm is utilized to generate heuristic exploration trajectories, which are then used for imitation learning to improve error recovery capabilities. Finally, a joint on‑and‑off policy learning framework is implemented, featuring high‑entropy trajectory sampling to enhance RL training efficiency and an error‑correction‑prioritized trajectory sorting strategy for effective error correction. Extensive experiments demonstrate the efficacy of JOP‑VLN, achieving success rates of 69.9% and 68.0% on the VLN‑CE R2R and RxR benchmarks, respectively, setting a new state‑of‑the‑art on R2R. Project page: https://qingrongh.github.io/JOP‑VLN.
Authors:Dwip Dalal, Shivansh Patel, Chahit Jain, Jeonghwan Kim, Utkarsh Mishra, Alex Baratian, Hyeonjeong Ha, Heng Ji, Svetlana Lazebnik, Unnat Jain
Abstract:
Finetuning a pretrained vision‑language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision‑language‑action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co‑training on web image‑text data, a common remedy, does not prevent this; it applies language and action losses to separate observations, leaving VLAs with language‑action misalignment that standard manipulation benchmarks do not expose. We propose Anchor‑Align, which augments BC with two objectives: Vision‑Language Anchoring distills layer‑wise representations from a frozen VLM copy to prevent this drift, while Language‑Action Alignment converts each action target into a discrete motion‑direction label and jointly trains language and action prediction on the same robot observation. On a physical xArm7 robot, across two widely used VLA architectures, Anchor‑Align improves real‑robot success on both (28% to 54% and 37% to 60%). At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long‑horizon control across LIBERO‑PRO, LIBERO‑Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io
Authors:Wisdom Dogah
Abstract:
Temperature scaling is the dominant post‑hoc calibration method in modern deep learning. Its theoretical justification rests on an assumption that is rarely stated explicitly: that ground‑truth labels are one‑hot and deterministic. In practice, labels are frequently soft, crowd‑sourced, or genuinely distributional, reflecting real disagreement among human annotators rather than annotation noise. We study whether temperature scaling retains its calibration properties when this assumption is violated, and whether any resulting degradation depends on model scale. Using CIFAR‑10H and ChaosNLI, two publicly available datasets with human‑annotated soft label distributions, we evaluate three model scales per modality under both hard one‑hot and soft distributional label targets. Across all nine configurations we find a positive soft‑label calibration gap: temperature scaling calibrated on hard labels consistently underperforms an oracle calibrated directly on soft labels, with Brier Score gaps ranging from 0.002 to 0.134. The gap grows monotonically with model scale in the vision domain and on the SNLI‑derived split of ChaosNLI, and is substantially larger in the language domain (mean gap 0.079) than in vision (mean gap 0.003). A scale‑ordering reversal on the MNLI‑derived split remains after matched‑domain training; we treat it as inconclusive for the scale hypothesis and attribute it primarily to near‑chance accuracy on that split. As a second post‑hoc baseline, multiclass isotonic regression yields the same qualitative conclusion: positive soft‑label gaps in all nine configurations, and larger gaps in language than in vision. These findings suggest that calibration protocols built on majority‑vote labels systematically misstate model reliability wherever label ambiguity is structural, with direct consequences for deployment in safety‑critical settings.
Authors:Kai Chen, Ming Dai, Wenxuan Cheng, Wankou Yang
Abstract:
Spatio‑Temporal Video Grounding (STVG) aims to retrieve the visual trajectory of a specific object from a video stream as described by a natural language expression. However, most advanced methods struggle to balance global context modeling with precise boundary localization. Due to the prohibitive computational costs of processing long videos, these approaches typically resort to low‑rate temporal downsampling and implicit motion modeling. This inevitably suppresses high‑frequency boundary cues and neglects the explicit inter‑frame dependencies required for precise boundary delineation. To address these limitations, we present ScanFocus, a novel coarse‑to‑fine framework that decouples the STVG task into a global spatio‑temporal scan and a local boundary focus. Specifically, we utilize a unified vision‑language fusion encoder combined with a lightweight Deformable Semantic‑Motion Fusion module to efficiently align multimodal features and generate coarse proposals. To recover the suppressed fine‑grained details, we introduce the Semantic‑Guided Temporal Aggregator (SGTA) in the refinement stage. By densely sampling around coarse boundaries, SGTA explicitly models short‑term temporal interactions under semantic guidance, capturing rapid motion changes for precise timestamp regression. Extensive experiments on three widely used benchmarks demonstrate the performance superiority of our proposed method over previous approaches. Code will be released at https://github.com/TenMinutes209/ScanFocus.
Authors:Jiwen Zhou, Xiang Liu, Mingming Li, Pengbo Mo, Jiao Dai, Honglei Lv, Jizhong Han, Songlin Hu
Abstract:
Recommender systems operate as Black‑Boxes, leaving users and regulators unable to steer their outputs toward specific intentions or audit their behavior. This lack of controllability, defined as the system's ability to respond to explicit guidance, remains an unaddressed dimension in existing evaluation paradigms. To fill this gap, we propose CtrlBench‑Rec, a collaborative multi‑agent framework for systematic assessment of controllability. We formalize three fundamental tasks: target content discovery, interest profile shaping, and popularity bias mitigation, which together measure steerability from explicit commands to implicit representation steering and finally to overcoming algorithmic biases.Extensive experiments on real‑world datasets and multiple recommendation models demonstrate that our framework effectively quantifies controllability and exposes critical system bottlenecks, most notably persistent resistance to guiding long tail content. CtrlBench‑Rec provides the first standardized toolkit for controllable recommendation research, algorithmic auditing, and user empowerment. Our code is released on https://github.com/caskcsg/CtrlBenchRec.
Authors:Jian Wang, Yang Yang, Ziheng Pan, Xiliang Zhu, Yuhan Zhang, Yanfeng Zhou, Dong Ni
Abstract:
Life‑limiting congenital anomalies require accurate prenatal diagnosis for appropriate clinical decision‑making. Prenatal ultrasound (US) examinations involve multiple anatomical planes, and diagnosis depends on identifying anatomical planes and selecting diagnostically relevant planes for each anomaly. Existing automated methods either rely on plane‑level annotations or aggregate heterogeneous images without explicitly modeling these diagnostic capabilities. We propose AnomExpert, a prototype‑driven framework for prenatal US anomaly diagnosis using only case‑level supervision. AnomExpert introduces learnable plane prototypes to organize unordered images into latent representations corresponding to anatomical planes without requiring plane annotations. A disease‑aware sparse selection mechanism further selects diagnostically relevant planes for each anomaly. Experiments on a multi‑center dataset of 3,654 cases show that AnomExpert consistently outperforms nine representative multi‑instance learning methods. Using a ViT‑small backbone, it achieves 86.9% accuracy and 84.2% F1‑score while maintaining parameter efficiency. These findings indicate that modeling anatomical plane identification and disease‑specific plane selection improves weakly supervised multi‑plane prenatal US anomaly classification. The code is available at https://github.com/TIanCat/AnomExpert.
Authors:Chun-Yi Kuan, Siwon Kim, Byeonggeun Kim, Suyoun Kim, Bo-Ru Lu, Qingming Tang, Ankur Gandhe, Hung-yi Lee, Chieh-Chi Kao, Chao Wang
Abstract:
Recent text‑to‑audio models generate high‑quality audio, but often fail to follow instructions involving multiple sound events and temporal order. This gap arises because existing evaluation and training signals mainly emphasize global similarity or perceptual quality, with limited supervision on instruction‑level correctness. We propose an instruction‑level framework that uses audio‑aware large language models (ALLMs) as fine‑grained judges to verify target event presence and temporal relations in generated audio. After validating ALLM judgments on benchmarks and through human verification, we use their feedback to construct preference pairs for direct preference optimization. We further introduce S3Bench, a narrative benchmark for evaluating multi‑event temporal instruction following. Experiments show that our method improves event completeness, temporal ordering, and joint instruction‑following accuracy across existing benchmarks and S3Bench, while maintaining audio quality.
Authors:Junning Lyu, Qizhi Guo, Xia Ning, Tao Song, Shaoming He
Abstract:
LiDAR‑inertial odometry (LIO) is a key component of autonomous navigation, but high‑dynamic driving exposes two coupled challenges: intra‑scan motion distortion and vibration‑contaminated inertial measurements. Most real‑time LiDAR‑inertial pipelines propagate the system state by integrating raw IMU measurements and then use the propagated trajectory for point cloud de‑distortion, thereby propagating inertial noise into both the corrected scan and the subsequent scan‑to‑map registration. This paper presents WNOJ‑LIO, a LiDAR‑IMU fusion framework based on a White‑Noise‑on‑Jerk (WNOJ) Extended Kalman Filter (EKF). WNOJ‑LIO employs a decoupled WNOJ prior on \R^3 × \SO(3) for state prediction and treats the IMU as a high‑frequency measurement source rather than the driver of state propagation. The resulting posterior state history is then used for LiDAR scan de‑distortion and subsequent point‑to‑plane LiDAR updates. The decoupled process model enables closed‑form covariance propagation, thereby bridging the gap between batch WNOJ Gaussian process (GP) trajectory priors and recursive filtering. Simulation results demonstrate improvements in acceleration and angular‑velocity denoising, scan de‑distortion, and localization accuracy over a FAST‑LIO‑style baseline. Real‑world experiments were conducted using an autonomous racing car on four driving segments with maximum speeds ranging from 53 to 208~km/h, covering a wide range of vehicle vibration levels. The experiments further validate the proposed method and provide a comprehensive evaluation of its performance in estimating acceleration, angular velocity, body‑frame linear velocity, attitude, and position under highly dynamic driving. The source code of WNOJ‑LIO is publicly available at https://github.com/LvJohny/wnoj‑ekf‑lio.git.
Authors:Xiaodong Liu, Michael Xu, Jack W. Stokes, Paul Smolensky, Doug Burger, Jianfeng Gao
Abstract:
Generative Flow Networks (GFlowNets) offer a promising alternative to reward‑maximizing reinforcement learning (RL) for large reasoning models, encouraging diverse reasoning paths by matching reward distributions rather than collapsing to dominant modes. Recent work shows promise on math and code, but scaling GFlowNet‑style RL to modern post‑training pipelines remains difficult: as model size, rollout horizon, reward noise, and distributed‑systems complexity grow together, a learned prompt‑conditional partition function becomes a source of gradient instability and engineering overhead rather than a useful normalizer. Through systematic analysis, we find that the learned partition function, previously treated as essential, can be replaced by an in‑batch Monte Carlo estimate computed from the rollout group already required for training. We propose GFlowRL, a streamlined GFlowNet‑style RL algorithm that removes the auxiliary partition network entirely while preserving the reward‑distribution‑matching objective, completed by two stabilizers: importance‑sampling correction for rollout/trainer drift and asymmetric flow‑gap clipping for outlier residuals. GFlowRL exceeds all counterparts on math, code, and adversarial red‑teaming benchmarks, reaching a Codeforces rating of 2048 at the 14B scale (within 25 Elo of o3‑mini) and attaining the highest average ASR@1 on AdvBench and HarmBench, outperforming the previous SOTA multi‑turn attacker in a regime where FlowRL, a prior GFlowNet‑style method, diverges. The same recipe transfers to all evaluated MoE configurations up to 235B parameters, where FlowRL again fails to converge. To our knowledge, GFlowRL is the first GFlowNet‑style RL algorithm to scale stably across both dense and sparse architectures. Code will be at: https://github.com/microsoft/gflowrl
Authors:Junlong Shen, Xingyu Li
Abstract:
Predictive Coding (PC) offers a biologically motivated alternative to backpropagation via local weight updates, yet routing error between layers still relies on an autograd Jacobian‑transpose (J^\top) product ‑ the last non‑local operation in PC. We show that this dependency is largely avoidable. For any layer f(x)=\mathrmAct(\mathrmNorm(L(x))) with frozen normalization statistics, the exact J^\top factors into three locally available terms, J^\top v = L^\top(s \odot σ'(z) \odot v), where σ' is the activation derivative, z is the pre‑activation, and s=γ/σ_\mathrmrun is the normalization gain. Prior weight‑feedback methods omitted both corrections; restoring them closes the transport gap for this layer class. Locality here holds up to three assumptions, which we state upfront: weight symmetry (L^\top mirrors the forward operator, as assumed by all PC), a soft spectral‑norm control that is not synapse‑local, and a nearest‑neighbour approximation for MaxPool. Substituting the identity into PC yields WF‑Act‑PC, which removes the autograd backward pass from error transport. On CIFAR‑10/100 (50 epochs, 5 seeds), WF‑Act‑PC is the only PC method whose accuracy improves with depth, surpassing iPC ‑ the strongest classical PC baseline ‑ by 2.7‑22.3 pp on CIFAR‑10. With both methods tuned per architecture, it matches or exceeds a comparably‑tuned backpropagation baseline on the deeper CIFAR‑10 architectures (VGG‑9: 93.57% vs. 92.43%; ResNet‑18: 92.76% vs. 91.54%) and on the harder Tiny‑ImageNet benchmark, while trailing tuned BP on the deeper CIFAR‑100 VGG cells. Our WF‑Act‑PC implementation is publicly available at https://github.com/jlshen025/pcax
Authors:Michinori Shimoji
Abstract:
Discourse data are the primary empirical basis of grammar writing in field linguistics, but producing interlinearized text is notoriously expensive ‑ on the order of one hour of work per minute of recording. For endangered languages, where the time remaining to verify analyses with native speakers is itself limited, automating parts of the interlinearization workflow has direct documentary value. We implement a full neural annotation pipeline (morpheme segmentation, POS tagging, glossing) for Irabu Ryukyuan using deliberately small, transparent BiLSTM‑CRF models, and evaluate it under a realistic hard constraint: approximately one hour of fully annotated discourse as the entire supervised resource. Two factors of the annotation itself are manipulated: its richness (with or without a POS tier) and its quantity (training budgets from 6 to 47 minutes). Gold POS improves grammatical glossing by +4.4 (SD 0.7) points (significant in all 5 seeds), and the gain grows as data shrink (+11.6 points at a quarter of the data); a POS tier more than halves the amount of glossed data needed to reach a given accuracy. In a fully automatic pipeline this gain is not yet realized: the tagger still errs on 12% of morphemes, and an incorrect POS misleads the glossing model more than no POS at all. The value is latent rather than lost: degrading gold POS with controlled noise shows the gain returning as tagger accuracy rises, with break‑even near our tagger's current 88% and +1.6 to +3.2 points recovered at 92‑96%. We conclude with a concrete recommendation for documentation practice: annotate quadrilinearly ‑ text, POS, gloss, translation.
Authors:Tessa Cannon, Michel Tsamados, Petru Manescu, Thomas Newman, Christian Haas, Veit Helm, Weibin Chen, Randall Scharien
Abstract:
Accurate estimation of landfast sea ice roughness is critical for climate modeling and safe Arctic over‑ice travel, yet existing approaches rely on costly airborne surveys or sparse in‑situ measurements, limiting spatial coverage and operational scalability. Here we show that high‑resolution sea ice topography can be reconstructed directly from optical satellite imagery using a conditional diffusion framework. Our approach, RoughNet, learns to map 10 m Sentinel‑2 multispectral images to locally normalized 1 m surface elevation residual fields, enabling fine‑scale roughness characterization from widely available satellite data. Trained on airborne LiDAR data from two Arctic regions and evaluated on an unseen third Arctic region, the model generalizes across diverse ice conditions and partially reproduces small‑scale topographic structure. The best‑performing model achieves an out‑of‑domain root mean squared error of 9 cm while preserving the statistical and spectral properties of the underlying roughness field. These results demonstrate that generative diffusion models can recover physically meaningful surface structure from optical imagery alone, providing a scalable pathway for high‑resolution sea ice mapping and roughness estimation in data‑sparse environments.
Authors:Ann-Kareen Gedeus, Jack Good, Nadine Wagener, Angelique Taylor
Abstract:
Embodied conversational agents (ECAs) need effective empathic grounding to foster social support and engagement. Expanding into emotional domains, ECAs now use Large Language Models (LLMs) and multimodal human‑agent interactions to enhance their capabilities. Yet, understanding the impact of backchanneling modalities on young adults and their gender remains limited. We introduce TANDE, an LLM‑powered ECA designed for emotional conversations with young adults, a population experiencing mental, personal, and social issues with limited tools to address them. In a within‑subjects study with N=36 young adults, we explore nonverbal and combined verbal‑and‑nonverbal backchanneling modalities on rapport, empathy, and engagement and isolate for gender differences. Our research shows the importance of nuanced backchanneling cues with emotional ECAs with young adults, showing a preference for nonverbal cues. We derive design implications for more effective ECAs for emotional support and well‑being in young adults. The code is available at https://github.com/Cornell‑Tech‑AIRLab/TANDE.
Authors:Luiz F. B. F. Martins, Rodrigo W. Pisaia, Matheus M. Girardi, Isabella Berkembrock, João A. Almeida, André G. Hochuli, Rayson Laroca, Alceu S. Britto
Abstract:
We present an audio‑text system for the Ambivalence/Hesitancy Video Recognition Challenge of the 11th ABAW Competition. The method excludes visual frames and represents each video as overlapping 5‑second windows aligned with transcript timestamps. Each window combines a 320‑dimensional prosodic audio descriptor, a 768‑dimensional emotion‑oriented RoBERTa embedding, and 74 handcrafted features capturing uncertainty, hedging, and attitudinal conflict. Audio and text are fused via temporal cross‑attention, while support features are injected prior to gated multiple‑instance learning (MIL) pooling to modulate the window's importance. Predictions from five independently initialized models are averaged. On the labeled public development set, the ensemble achieved an average precision of 0.875 and a macro‑F1 of 0.72. Our source code is publicly available at https://github.com/Liga‑de‑IA‑PUCPR/abaw‑11‑ah‑challenge/.
Authors:Yuxin Huang, Ziming Hong, Mingming Gong, Wanyu Wang, Jing Zhang, Tongliang Liu
Abstract:
Recent diffusion‑based video generation models have enabled high‑quality personalized video customization through both tuning‑based pipelines, which fine‑tune a video diffusion model, and reference‑based pipelines such as image‑to‑video generation. However, these capabilities raise serious concerns about personal privacy, identity ownership and intellectual property protection. Existing anti‑customization works focus on protecting images, while protection for videos against both reference‑ and tuning‑based customization remains largely underexplored. Protecting videos in this setting raises three challenges: (i) Image‑level perturbations, optimized frame by frame, cannot survive temporal compression by 3D video VAE. (ii) A video‑level perturbation optimized on a single video is vulnerable to temporal editing and fails to protect unseen videos. (iii) Temporally inconsistent perturbations are not robust to temporal attacks. To address these challenges, we propose Temporally Consistent Universal Adversarial Perturbations (TC‑UAP), the first protection method against both reference‑ and tuning‑based video customization. TC‑UAP optimizes an identity‑level multi‑frame UAP over sliding windows from multiple videos, accounting for local temporal dependencies induced by temporal compression in video VAE and enabling a single perturbation to protect unseen videos of varying lengths. Moreover, we introduce intrinsic temporal modeling and an extrinsic surrogate temporal‑attack loss, which make the perturbation temporally consistent and robust to unseen temporal attacks. Empirically, quantitative and qualitative results show that TC‑UAP achieves the strongest identity protection compared with existing methods under both reference‑ and tuning‑based video customization, and remains robust under multiple unseen temporal attacks.
Authors:Tyler Ward, Abdullah Imran
Abstract:
The Segment Anything Model (SAM) has demonstrated strong generalizability across a variety of segmentation tasks. However, SAM often struggles in situations where the target to be segmented is ambiguous. This poses a problem in medical imaging, where accurate delineation of targets such as tumors is vital, but even expert radiologists can disagree on the appropriate boundary for a target. Addressing this, we propose SARFA (Segment Anything with Radiomic Feature Alignment), a novel framework for improved medical image segmentation. Via probabilistic prompting, SARFA generates a diverse set of plausible masks for each input image and optimizes them with a radiomics‑driven training objective based on Fréchet Radiomic Distance (FRD) and Direct Preference Optimization (DPO). By minimizing the FRD between masked predicted and ground truth regions within each image, SARFA encourages segmentation outputs whose anatomical and textural characteristics align with clinically meaningful ground truth representations, without relying solely on pixel‑level overlap. Evaluated on computed tomography (CT) and magnetic resonance imaging (MRI) benchmarks, SARFA outperforms existing ambiguous segmentation methods, demonstrating the effectiveness of radiomic feature alignment and DPO‑style candidate mask ranking as a training objective. Our code is available at https://github.com/tbwa233/SARFA.
Authors:Jae Joong Lee
Abstract:
Benchmark accuracy in video large language models (LLMs) is often treated as evidence of visual understanding. We audit this assumption across twenty models spanning 2‑78B parameters and ten architecture families. We introduce the Visual Dependency Gap (VDG), the difference in per‑question correctness between original‑video and black‑screen conditions. Paired McNemar tests on MVBench show that accuracy and visual dependency are separable: models differ on original video (p = 0.0003) but not on black screens (p = 0.53). Across models, task‑type rankings are stable: Attribute Perception is strongly visual, whereas Temporal Reasoning approaches the language‑only baseline. A diagnostic ladder from black screen to single frame, shuffled frames, and original video reveals that frame diversity supplies most of the visual benefit, while temporal order contributes near‑zero accuracy across sixteen open‑weight models. An ablation from 0.5 to 24 FPS rules out sparse sampling as the cause. H.264 experiments further show that stable aggregate accuracy conceals bidirectional question‑level answer flips. The diagnostic also generalizes to four API‑accessed models, whose VDG values range from 0.025 to 0.315. These results motivate VDG as a standard audit for whether video benchmarks measure visually grounded capability. Code is available at https://github.com/JaeLee18/accuracy‑without‑grounding.
Authors:Marcus J. Min, Mike He, Zhaoyu Li, Zixuan Yi, Sharad Malik, Aarti Gupta, Xujie Si, Osbert Bastani
Abstract:
Autoformalization translates informal natural language into formal, machine‑verifiable languages. While most work focuses on individual statements, real formalization efforts are inherently theory‑level: they require an entire web of axioms, definitions, and lemmas before target theorems can even be stated. In this position paper, we argue for theory‑level autoformalization: formalizing complete theories, including all their inter‑dependencies, as structured libraries. We examine the significance of this shift, address alternative views, identify open challenges, and propose three promising paths forward. Our survey of autoformalization is available at https://github.com/marcusm117/Awesome‑Autoformalization.
Authors:Ruhan Wang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Yue Yu, Junyao Yang, Kishan Panaganti, Haitao Mi, Dongruo Zhou, Leoweiliang
Abstract:
The capability of a modern AI agent depends not only on its foundation model but also on its harness, which constructs prompts, manages state, invokes tools, and coordinates execution. As models, APIs, environments, and requirements evolve, the harness must be continually modified. Before such a change can be made, a developer or coding agent must identify all code locations that implement the target behavior. This is difficult because production harnesses are large, tightly coupled, and behaviorally distributed, while modification requests describe what the system should do and repositories are organized by files and modules. Code search, repository indexing, and long‑context processing ease inspection, but still leave this behavior‑to‑code mapping to be recovered by hand. Behavior localization is therefore a central bottleneck in harness evolution. We introduce the Harness Handbook, a behavior‑centric representation synthesized automatically from a harness codebase via static analysis and LLM‑assisted structuring, linking each behavior to its corresponding source. We also introduce Behavior‑Guided Progressive Disclosure (BGPD), which guides agents from high‑level behaviors to relevant implementation details and verifies candidate locations against the current source. On diverse modification requests from two open‑source harnesses, Handbook‑Assisted planning improves behavior localization and edit‑plan quality while using fewer planner tokens, with the largest gains on scattered sites, rarely executed paths, and cross‑module interactions. Evolving complex agentic systems thus depends not only on generating edits, but also on determining where those edits should be made.
Authors:Ben Maman, Frank Zalkow, Hans-Ulrich Berendes, Paolo Sani, Christian Dittmar, Meinard Müller
Abstract:
Recent diffusion‑based generative models have achieved strong results in domain‑specific audio generation tasks such as speech, singing, and instrumental music synthesis. However, these models are typically specialized and do not generalize well to mixed or intermediate audio types. In this work, we adapt a diffusion‑based model originally designed for multi‑instrument music synthesis to voice conversion, covering both speech and singing within a unified framework. Specifically, we extend musical note‑based conditioning to include phonetic posteriorgrams (PPGs) and pitch contours, and reinterpret timbre conditioning as speaker or singer identity via feature‑wise linear modulation. Experiments show that the adapted model matches or surpasses a dedicated voice conversion system in terms of naturalness and performer similarity, while maintaining accurate pitch control across speech and singing. At the same time, we observe limitations in phonetic fidelity and a degradation in vocal quality when incorporating instrumental training data. Furthermore, we demonstrate that off‑the‑shelf feature extractors provide effective conditioning signals, enabling large‑scale self‑supervised training without manual annotations. These results highlight the potential of cross‑domain model transfer towards unified audio generation systems capable of handling speech, singing, and music. Qualitative samples can be found on our project page: https://benadar293.github.io/voice‑conversion
Authors:Huy Che, Hoang-Minh Trinh, Dinh-Duy Phan, Duc-Lung Vu
Abstract:
Face identification has achieved remarkable performance under normal conditions. Yet, its accuracy often degrades significantly when query faces are partially occluded, especially by facial masks. Existing re‑ranking approaches improve robustness by exploiting patch‑level similarities. Still, they often rely on costly, fine‑grained matching mechanisms, which limit their efficiency in large‑scale retrieval scenarios. In this paper, we propose MGFace, a mask‑gated face identification pipeline that predicts the mask status of a query face and conditionally routes the similarity computation accordingly. Specifically, MGFace distinguishes between masked and unmasked queries, applies global embedding matching to unmasked queries, and activates mask‑aware patch‑level re‑ranking only for masked queries. This design focuses on reliable upper‑face regions while avoiding unnecessary fine‑grained computation. Experiments on the extended LFW‑Mask dataset show that MGFace achieves over 80% identification accuracy with the FaceNet backbone and over 90% with the ArcFace backbone. Compared with a previous EMD‑based re‑ranking method, MGFace achieves better identification performance while reducing query time by approximately 20x. These results demonstrate the effectiveness of MGFace in improving masked‑face identification accuracy with low computational overhead. The source code is available at https://github.com/chequanghuy/MGFace.
Authors:Jeffrey S. Baggett, Huiya Yan
Abstract:
For integer vectors R,S let A(R,S) denote the class of (0,1)‑matrices with row sum vector R and column sum vector S. Its interchange graph G(R,S) has A(R,S) as its vertex set, two matrices being adjacent when they differ by a single 2 x 2 interchange. Brualdi asked whether G(R,S) is Hamiltonian for every R,S. We prove the stronger statement that G(R,S) is maximally Hamiltonian: Hamilton‑laceable when bipartite, and Hamilton‑connected when not. The proof is a structural induction on the number of matrices in the class, organized by the structure theory of interchange graphs. Deleting inactive lines and splitting invariant positions expresses any class as a Cartesian product, reducing the argument to the prime factors. The bipartite classes are products of complete transposition graphs; we settle them together, without induction, by proving they are paired 2‑disjoint‑path‑coverable and hence Hamilton‑laceable, using a recent theorem of Coleman, Fischberg, Gong, Harrington and Wong on paired disjoint path covers. The non‑bipartite classes divide into three cases: products assembled from smaller factors, a base of Johnson graphs and small classes, and the large prime classes, treated by a pivot‑and‑fiber construction whose line quotients are matroid base‑exchange graphs. The complete argument has been machine‑checked in the Lean 4 proof assistant from first principles together with seven cited results of the literature; the disjoint‑path‑cover results it imports are themselves proved within the formalization.
Authors:Ruize Xia
Abstract:
Sign language is a primary communication channel for millions of Deaf and hard‑of‑hearing people, yet text‑to‑signer video generation remains costly because video diffusion models are expensive to train and evaluate. This paper presents Text2Sign, a text‑conditioned diffusion model for short sign‑language clips that runs on a single NVIDIA L4 GPU. It combines a frozen vision‑language text encoder with a 3D encoder‑decoder and factorized spatiotemporal attention to reduce the cost of full‑video attention while preserving motion coherence. We compare convolution‑only and transformer‑style backbones, frozen pretrained and task‑specific text encoders, and factorized versus full attention. On a signer‑disjoint How2Sign split, the best short‑run ablation reaches a validation loss of 0.0648, while a longer‑run checkpoint reaches 0.00999. On a compact evaluation slice, the latter achieves an SSIM of 0.2403 \pm 0.0238, a PSNR of 15.11 \pm 0.42 dB, and temporal consistency of 1.0000 \pm 0.0000 using 8‑step DDIM sampling with a guidance scale of 5.0. It generates a 32‑frame, 64 × 64 clip in 12.60 seconds, or 2.54 frames per second, with peak inference memory of 3.12 GB. A held‑out denoising audit shows only weak prompt sensitivity: removing text increases late‑timestep loss from 0.9875 to 0.9891, while shuffled prompts perform similarly to correct prompts. Frozen text conditioning therefore improves short‑budget validation loss, but prompt‑specific separation remains limited. The system is restricted to low‑resolution, short clips and lacks expert linguistic evaluation, so it should be viewed as a single‑GPU research baseline rather than a complete sign‑language production system. Code is available at https://github.com/xiaruize0911/text2sign.
Authors:Guoxuan Chen, Chufeng Xiao, Haoran Yang, Siyue Xie, Binxiao Huang, Ming Zhang, Cheuk Him Chau, Xinyu Fu, Yingzhao Lian, Tom S. Y. Li, Jintao Lin, Bowen Dong, Zian Qian, Yuhao Liu, Yuxuan Hu, Weikang Shi, Bin Zou, Bowen Zheng, Haoxuan Che, Chang Chen, Yuyang He, Heyang Sun, Tianyu Huang, Chong Hou Choi, Cheng Gong, Han Shi, Haoli Bai, Xihui Liu, Hongsheng Li, Qifeng Chen, Chao Huang, Rui Liu, Chenyang Lei
Abstract:
We introduce Boogu‑Image‑0.1, an open‑source unified multimodal understanding and generation model family, comprising Base, Turbo, Edit, and Edit‑Turbo variants. It delivers competitive performance in high‑quality text‑to‑image generation, fast inference, instruction‑based editing, and bilingual (Chinese‑English) text rendering. Closed‑source multimodal systems like Nano‑Banana‑Pro and GPT‑Image‑2 achieve strong performance through system‑level integration rather than a single model, yet their internal practices remain largely undisclosed. In this work, we demonstrate that strengthening the understanding capability of the system, through a stronger multimodal encoder, agentic prompt rewriting, and related techniques, together with improvements in data quality, training pipelines, and agentic inference‑time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets. Comprehensive evaluations show that Boogu‑Image‑0.1 consistently matches or surpasses other open‑source models across standard benchmarks, and achieves results approaching leading closed‑source systems. Notably, this is accomplished with only 208.62 million unique images. The base model's theoretical training cost is only approximately \400K. We share practical discussions that we believe are valuable to the broader research community, and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified multimodal understanding and generation. Our code is available here: https://github.com/Boogu‑Project/Boogu‑Image.
Authors:Yang Qianl, Liu Xiany, Dai Daw, Chen Jing, Shen Xiaoj, Fu Kaiw, Tang Ming, Zou Dongl
Abstract:
ThinPrep Cytologic Test (TCT) enables early cervical cancer screening, but manual reading is time‑consuming and yields inconsistent diagnostic results among cytopathologists. Existing AI detection models perform poorly under real clinical conditions, primarily restricted by two key constraints: unbalanced spatial distribution of cell populations in TCT slides, and limited high‑quality annotated cytology data relying on professional pathologist labeling. To address these limitations, we propose a Cell‑Distribution Normalization (C‑Norm) method. By decoupling abnormal and normal cells from the original TCT images and re‑synthesizing them, this method ensures a uniform distribution of cell populations, thereby mitigating generalization degradation caused by distribution bias. Building upon this, we integrate the YOLOv12 framework with a DINOv3 module. This hybrid architecture leverages the advanced detection capability of YOLO models and the superior feature representations of DINOv3 to capture subtle morphological nuances essential for precise recognition of TCT images. Extensive experiments demonstrate that our proposed method achieves state‑of‑the‑art performance, significantly outperforming mainstream detection algorithms. The complete implementation is available at: https://github.com/ddw2AIGROUP2CQUPT/Cell‑Norm
Authors:Zhe Ren, Yimeng Chen, Dandan Guo, Guowei Rong, Tonghui Li, R. B. Xiong, Qingfeng Lan, Wenyi Wang, Li Nanbo, Yibo Yang, Mingchen Zhuge, Jürgen Schmidhuber
Abstract:
Self‑improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, from experience with minimal or even no human input. This survey frames modern self‑improving agents as adaptive systems that convert experience into accumulated capability gains. We offer a system‑level framework that represents a modern agent as a configuration coupling a foundation model with an operational scaffold of prompts, memory, tools, and control logic. Within this framework, self‑improvement is formalized as a self‑induced update operator that obtains and commits updates to model parameters or scaffold components. We organize prior work by update target and by the signals that drive change, then review applications and discuss evaluation, before closing with open problems and future directions. For convenience, we track technical updates on https://github.com/selfimproving‑agent/awesome‑Self‑Improving‑Agents.
Authors:Jean-Jacques Dubray
Abstract:
Can large language models write faithful formal specifications of real systems, and does it matter whether they write in a formal language they have seen rarely or in a mainstream language abundant in their training data? We study this on SysMoBench, which grades a generated specification in four phases, the decisive one replaying execution traces captured from the running system. We add JS‑SAM, its first non‑formal backend, in which a specification is executable JavaScript written in the SAM pattern, a pattern whose semantics mirror TLA+, and run a controlled comparison that separates three variables an ordinary head‑to‑head entangles: the language, the specification contract (the shape the model must fill), and the prompt. The study spans four frontier models and three systems (an operating‑system spinlock, a distributed lock service, and the Etcd Raft consensus implementation), with counterexample‑driven repair. Three findings emerge. First, conformance against the real system is the only phase that discriminates among models; internal consistency is inexpensive to satisfy, and a specification that looks right is not thereby right. Second, once the comparison is drawn like for like, the specification contract, not the language, governs fidelity: JavaScript in the shape of the TLA+ transition relation is as faithful as TLA+. Third, a minimal contract carries transcription but not semantic derivation: at consensus scale the difficulty becomes understanding the protocol, which no contract shape and no language supplies. We frame executable JavaScript as a checkable specification substrate that complements, rather than replaces, the verification TLA+ provides, and present the study as a case study.
Authors:Dominik Schwarz
Abstract:
Context can change whether a request is harmful without changing its topic or surface form. We ask whether residual‑stream probes distinguish harmful requests from surface‑matched benign controls at a useful operating point. Across three 7‑8B model families, an activation sensor blocks 95.5‑97.7 percent of judge‑classified compliant attacks in a taxonomy‑selected set. It also blocks 59.6‑68.4 percent of XSTest prompts. A fully disjoint audit reconstructs near‑ceiling source‑contrast AUROC (0.996‑0.999), but fixed transfer to matched pairs is weaker: 0.656‑0.819 on the guard‑selected Twin‑n70 subset and 0.590‑0.690 on the full Twin‑n163 cohort. We test ten axes on the reference family and seven across all families with leakage, hold‑out, and permutation controls. On Twin‑n163, no axis evaluated without direct pair‑boundary fitting reaches the specified numerical threshold. Requiring persistence on that full cohort was added at analysis time. A separately specified 24B/32B extension gives the same result. Pair‑trained classifiers weaken under category and generation‑batch hold‑out and false‑block 79.6‑100 percent of XSTest at 95 percent in‑corpus TPR. At the tested read points, these activation scores behave as broad‑risk detectors rather than standalone context adjudicators.
Authors:Junjie Yin, Xinyu Feng
Abstract:
Large language model (LLM) agents increasingly automate multi‑step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires. They often follow a maximum‑context‑first strategy‑‑re‑reading files and dependencies they have already seen‑‑turning a one‑line edit into a small code‑base audit. We argue the missing capability is task‑aware execution‑scope estimation: judging a task's difficulty, the information it truly needs, and the shortest reliable path before committing budget. We formalize minimum‑sufficient execution and the Agent Cognitive Redundancy Ratio (ACRR), and propose E3 (Estimate, Execute, Expand): the agent estimates an initial operating point, executes a minimum viable path, and expands scope only when verification fails. On MSE‑Bench‑‑a deterministic benchmark of 121 edits in a capability‑controlled simulator‑‑E3 matches the strongest baseline's 100% success while cutting cost by 85%, tokens by 91%, and inspected files by 92%, and further beats a strong adaptive retrieval baseline by 16%; the gains survive held‑out instruction wording and essentially every cost weighting. A companion real‑model harness (LLM‑Case) corroborates the effect on a live gpt‑4o agent editing a real open‑source library, with every candidate patch graded by actually running the project's real pytest suite against a measured oracle: the over‑reading is milder but real, and E3 is the leanest and fastest policy at comparable task success‑‑its one shortfall a provider rate‑limit, not a wrong edit. We frame this as a controlled probe of execution redundancy, not a measurement of any deployed agent, and position task‑aware execution as a step toward engineering‑grounded AI (EGAI)‑‑agents whose effort is anchored in the engineering reality of the task. We release the framework and benchmark.
Authors:Hongru Cai, Yongqi Li, Ran Wei, Wenjie Li
Abstract:
Large Language Model (LLM) agents have moved beyond generating responses to executing multi‑step tasks by calling tools, observing the results, and iteratively deciding the next action. Most agent systems run on desktops or servers, which support tool use and task automation. Mobile devices are also important agent environments because they are widely accessible and contain users' data, sensors, and daily‑use applications. Existing mobile agents mainly operate smartphones through graphical user interface (GUI) actions such as tapping, swiping, and typing, which often form long, interface‑dependent sequences, cannot directly access device capabilities, and make execution boundaries difficult to define. We present PalmClaw, an open‑source agent framework that runs natively on mobile phones and manages the sessions, memory, skills, tools, and agent loop directly on the device. PalmClaw exposes device capabilities as device tools with explicit arguments, structured results, and clearly defined execution boundaries. This design enables agents to use mobile capabilities directly while keeping each action explicit and controlled. Experiments show an 11.5% relative improvement in task success and a 94.9% reduction in completion time over the strongest baseline, with lower setup burden and traces illustrating how execution boundaries are applied. Code is available at https://github.com/ModalityDance/PalmClaw.
Authors:Héctor Carrión, Narges Norouzi
Abstract:
Dermatological practice routinely involves measuring and tracking lesion size, morphology and texture, as critical components of wound or skin cancer screening, monitoring and diagnosis. To accomplish this task, practitioners often image the skin surface with commonly available off‑the‑shelf camera sensors. This has led to an overwhelming research focus on 2D methods while these objectives naturally benefit from 3D information. In this paper, we demonstrate that dense monocular 3D reconstructions, metric scale measurements and rich surface normal texture estimates are achievable for both dermoscopic and macroscopic cases without the need for additional hardware or multiple captures. We present DermDepth, the first single‑view metric scale 3D model for the dermatological domain and D‑Synth, the first synthetic dermoscopic dataset with pixel‑perfect 3D information. Our experiments show training DermDepth on D‑Synth corrects metric scale error from over 16x to under 1.1x for real dermoscopic data, while preserving geometric quality and increasing texture richness. Fine‑tuning on a small amount of real clinical samples generalizes our method across three real‑world benchmarks spanning the few mm to hundred cm range, diverse skin‑tones, chronic wound cases and produces measurements broadly consistent with disease size reported in medical literature. All code, data and models are available at https://github.com/hectorcarrion/dermdepth.
Authors:Zhao Yang, Yinan Shi, Mingyuan Yao, Wenyao Xue, Yawei Jueluo, Longjun Liu
Abstract:
Vision‑language action (VLA) models increasingly adopt chunked action heads to satisfy real‑time constraints; however, this introduces boundary jitter: overlapping regions between consecutive chunks often yield inconsistent predictions, degrading temporal coherence and the task success rate. Existing methods, such as inference‑time blending, merely reweight mismatched proposals without correcting underlying errors, leading to residual accumulation under biased or noisy histories. We propose ChunkFlow, a seam‑aware training‑and‑execution framework for chunked policies that aligns chunk structure with boundary execution. It partitions each chunk into frozen, editable, and future zones, applies deterministic overlap blending at execution, and trains raw predictions with seam and first‑ and second‑order continuity losses. History corruption and scheduled sampling improve robustness to executed‑history errors, while an AWAC fine‑tuning stage adapts the policy without removing these structural regularizers. Under mild smoothness assumptions, pre‑blending seam discrepancies provably decay with increasing overlap. Experiments on CALVIN, LIBERO, and real robots show an improved success‑stability trade‑off with low‑latency inference. Project page: https://cytoderm‑ai.github.io/chunkflow.
Authors:Héctor Carrión, Narges Norouzi
Abstract:
Accurate dermatological diagnosis naturally necessitates equitable performance across diverse populations, yet a systematic lack of expertly annotated images, especially for underrepresented skin tones and rare diseases, impedes progress toward measurably fair methods. We introduce cgDDI (Controllable Generation of Diverse Dermatological Imagery), a hybrid framework that (1) synthesizes realistic healthy skin samples without disturbing other input properties, (2) maps single‑sample rare lesions onto novel skin‑tones and locations non‑parametrically, and (3) allows for efficient parametric generation with as few as 10 training samples. The framework supports both human and automated segmentation masking, enabling scalability to datasets without pre‑made lesion masks. We grow a 656‑image dataset by more than 400x and validate across two datasets: biopsy‑confirmed Diverse Dermatology Images (DDI) and expert‑verified Fitzpatrick17k (F17k). On the DDI benchmark, we achieve malignancy classification accuracy of 86.4% under synthetic‑only training and 90.9% state‑of‑the‑art performance with real data fine‑tuning, alongside leading fairness metrics. Cross‑dataset experiments show +13.9% accuracy improvements on unseen F17k data despite minimal disease overlap. We openly release 266k+ synthetic images, code, and generative models to further support fairness research at https://github.com/hectorcarrion/ControllableGenDDI.
Authors:Minh Hoang Nguyen
Abstract:
Recommender‑system research for Vietnamese remains limited by the absence of a public, well‑documented hotel interaction resource. Building such a resource is challenging for three reasons: cross‑platform hotel names must be reconciled before interactions are comparable; quality must be audited with reproducible metrics rather than ad hoc cleaning; and public release must preserve privacy while remaining benchmarkable under realistic cold‑start conditions. We introduce ViHoRec, a quality‑controlled Vietnamese hotel recommendation dataset of 18,267 interactions between 6,832 users and 560 hotels, crawled from Booking.com, Traveloka, and Ivivu. Our contributions are: (i) a reproducible construction pipeline with cross‑platform entity resolution and quantitative quality control; (ii) a privacy‑preserving release with HMAC pseudonyms; and (iii) a public cold‑start benchmark with temporal leave‑last‑one‑out split, data‑centric ablations, and dependency‑free baselines. On the public split, learned models degrade sharply for users with short histories (BPR‑MF Recall@10: 0.065 vs. 0.120), while UserKNN remains strongest overall, establishing ViHoRec as a sparse, cold‑start‑dominated testbed for low‑resource recommendation. All data are publicly available at https://github.com/MinhNguyenDS/ViHoRec.
Authors:Adam Schmidt, Mert Asim Karaoglu, Zijian Wu, Jiaming Zhang, Yuxin Chen, Tim Salcudean, Ho-Gun Ha, Minkang Jang, Kyungmin Jung, Ihsan Ullah, Hyunki Lee, Suresh Guttikonda, Sarah Latus, Alexander Schlaefer, Xinkai Zhao, Yuichiro Hayashi, Masahiro Oda, Takayuki Kitasaka, Kensaku Mori, Peng Liu, Chenyang Li, Stefanie Speidel, Aoife Gardiner, Agostino Stilli, Danail Stoyanov, Francisco Vasconcelos, Anwesa Choudhuri, Meng Zheng, Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Terrence Chen, Ziyan Wu, Alexander Ladikos, Omid Mohareri
Abstract:
Point tracking in surgery is crucial to enable applications in downstream tasks such as segmentation, 3D reconstruction, virtual tissue landmarking, autonomous probe‑based scanning, and subtask autonomy. This paper introduces the 2025 iteration of a point tracking challenge to address this, wherein participants submit their algorithms for quantification. Their algorithms are evaluated using a dataset named surgical tattoos in infrared (STIR), with the challenge named the STIR Challenge 2025 (STIRC2025). The STIR Challenge 2025 comprises two quantitative components: accuracy and efficiency. The accuracy component tests the accuracy of algorithms on in vivo and ex vivo sequences. The efficiency component tests algorithm inference latency. The challenge was conducted as a part of MICCAI EndoVis 2025, and seven teams participated in this challenge. In this paper we summarize the challenge results and participant methods. The challenge dataset is available at: https://zenodo.org/records/20191078, and the code for baseline models and metrics calculation is available here: https://github.com/athaddius/STIRMetrics
Authors:Yunzhou Li, Jiesi Hu, Yanwu Yang, Hanyang Peng, Chenfei Ye, Jianfeng Cao, Yixuan Yuan, Ting Ma
Abstract:
Medical image segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fragmented by prompt paradigms and spatial dimensions. Visual in‑context learning, interactive segmentation, and language‑guided segmentation are typically handled by paradigm‑specific models, while 2D and 3D images are also modeled separately. Such isolation prevents heterogeneous annotations and data from being jointly absorbed by a single scalable model and limits cross‑paradigm knowledge transfer. To address this bottleneck, we propose UniMedSeg, a Transformer‑centric universal segmentation framework that maps visual examples, geometric interactions, language instructions, and 2D/3D images into a shared sequence space, enabling heterogeneous medical supervision to be jointly learned through a unified in‑context interface without prompt‑ or dimension‑specific branches. To overcome the long‑sequence memory bottleneck caused by visual contexts, we introduce Decoupled Split Attention, which reduces attention complexity to linear while preserving hardware‑friendly computation and focused context‑target interaction. Extensively trained and evaluated on a large corpus curated from 27 public datasets, UniMedSeg achieves state‑of‑the‑art performance across visual in‑context, interactive, and language‑guided segmentation without task‑specific fine‑tuning, demonstrating strong generalization on diverse held‑out tasks. The code and model weights are publicly available at https://github.com/Lii1228/UniMedSeg
Authors:Ziyi Wang, Xumin Yu, Yongming Rao, Yonggen Ling, Yunheng Li, Oran Wang, Mingqi Gao, Yuchen Zhou, Yves Liang, Zuyan Liu, Yani Zhang, Rui Huang, Xiaoran Xu, Bowen Yuan, Yifu Yuan, Xu Tan, He Zhang, Yufei Huang, Shenghao Zhang, Hongsheng Wu, Han Hu, Zhengyou Zhang
Abstract:
Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy‑Embodied‑VLM‑1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre‑training stage onward, we define an action‑centric capability taxonomy comprising three progressive dimensions: Action‑Relevant State Understanding, Action‑Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre‑training and post‑training. To deliver strong physical‑world understanding and interaction capabilities while supporting latency‑sensitive deployment, we build our model on the Hy3‑A3B language backbone and the Hy‑ViT2 vision encoder. Its efficient Mixture‑of‑Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy‑Embodied‑VLM‑1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical‑world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6‑A3B and Cosmos 3. Compared with the previous‑generation Hy‑Embodied‑0.5 MoT‑2B, Hy‑Embodied‑VLM‑1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous‑generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy‑Embodied‑VLM‑1.0 also demonstrates strong performance on embodied agentic tasks requiring multi‑turn interaction and long‑horizon reasoning.
Authors:Zhiyu He, Zecheng Zhao, Tong Chen, Zi Huang, Yiqun Liu, Min Zhang
Abstract:
Video thumbnails are a key factor for attracting user clicks on video platforms, and are increasingly supported by automation. However, existing thumbnail generation methods typically produce generic results shared across users, overlooking the diversity of individual preferences. We therefore introduce personalized video thumbnail generation, a novel task that aims to create thumbnails tailored to user‑specific preferences. It is challenging in two aspects: (i) identifying visual anchors (i.e., key frames) from each video to guide the generation, which requires a balance between personalization and informativeness that existing highlight detection methods fail to achieve; and (ii) generating personalized thumbnails that are both visually coherent and faithful to the original video. As a response, we propose a two‑stage framework that tightly couples preference‑aware retrieval with controllable generation. In the first stage, a personalized highlight retriever captures fine‑grained user‑video interactions and incorporates video semantics through summarization, enabling the selection of diverse visual anchors aligned with both user preferences and video contexts. In the second stage, a VLM‑guided diffusion pipeline transforms these anchors into thumbnails by extracting and injecting semantically grounded visual cues, improving personalization while preserving visual coherence and fidelity. Experiments on two public datasets show our method delivers state‑of‑the‑art performance compared with both retrieval‑based and generative baselines. A user study further demonstrates improved click preference, highlighting its effectiveness in enhancing user engagement. The code is available at https://github.com/hezy18/PVTG.
Authors:Peter R. D. van der Wal, Nicola Strisciuglio, George Azzopardi
Abstract:
Vision Transformers (ViTs) have demonstrated remarkable performance in computer vision tasks. However, their self‑attention mechanism often diffuses focus across background regions, relying on spurious correlations rather than object‑relevant cues. Inspired by inhibitory mechanisms observed in biological vision systems, we propose the Inhibited Self‑Attention (ISA), a novel self‑attention that integrates inhibitory signals to enhance feature selectivity and suppress spurious responses. In contrast to conventional self‑attention, which relies solely on positive attention values due to softmax normalization, our approach retains and utilizes negative attention scores to suppress irrelevant features and sharpen focus on objects of interest. Experiments across multiple datasets, including ImageNet‑1k and COCO, and several robustness benchmarks demonstrate that ISA enhances object‑centric selectivity, reduces shortcut reliance, and improves out‑of‑distribution generalization. Our analysis of relevance maps confirms that ViTs with ISA exhibit sharper, more localized focus on object‑relevant regions while reducing distractions from non‑relevant (background) features, enabling more reliable models. We release our code at https://github.com/prdvanderwal/inhibited‑self‑attention
Authors:Zhenwen Miao, Honglin Wang, Mingheng Mi
Abstract:
As LLM technology advances, the space of model families, compute hardware, quantization schemes, parallelization strategies, and specialized optimization kernels continues to expand, sharply increasing the code complexity and maintenance cost of general‑purpose inference frameworks. Conventional software engineering uses multiple layers of abstraction to support diverse application scenarios, but these abstractions also increase system complexity and may introduce additional performance overhead. This paper presents metainfer, an 'LLM‑as‑Compiler' approach in which users specify only the runtime constraints of an inference program. An LLM‑driven multi‑agent collaboration system, coupled with a contract knowledge base, then automatically generates a compact customized inference framework that satisfies these constraints. We evaluate metainfer from three perspectives: the effect of source‑code reference, the runtime behavior and performance profile of engines generated under the zero‑reference constraint on CKB‑covered targets, and knowledge‑base evolution for new model and platform scenarios. The results show that metainfer organizes generation constraints, validation feedback, and knowledge consolidation into a continuous closed loop, enabling runnable customized inference solutions to be generated from explicit knowledge. The code is publicly available at https://github.com/MetaInfer/MetaInfer.
Authors:Nguyen Minh Tri, Hoang Khuong Duy, Huynh Cong Viet Ngu
Abstract:
Reconstruction‑based methods are a cornerstone of unsupervised image anomaly detection, but they remain vulnerable to \emphoutlier leakage, where standard mean squared error (MSE) loss drives the model to faithfully reconstruct anomalous patterns. We propose a Non‑linear Reconstruction Loss that applies a sigmoid‑based squashing function to suppress high‑magnitude features, preventing outliers from dominating optimization while preserving sensitivity to normal patterns. In addition, we introduce a statistical calibration scheme that selects the scaling factor k from the confidence interval (CI) of the normal feature distribution, enabling data‑driven control of the suppression strength. Our approach achieves competitive or superior anomaly detection performance compared to state‑of‑the‑art methods, reaching 99.0% Image‑AUROC and 97.3% Pixel‑AUROC on MVTec‑AD, and 95.3% Image‑AUROC and 99.0% Pixel‑AUROC on VisA. These results indicate that non‑linear gradient suppression is an effective mechanism for mitigating outlier leakage and improving anomaly localization in unified industrial inspection settings. The implementation is available at https://github.com/mintii13/Statistical‑Non‑linear‑Reconstruction‑Loss.git.
Authors:Zhongwei Ren, Yunchao Wei, Yao Zhao, Weibo Gong, Xiao Liu, Anran Wang, Xiangtai Li, Xiaojie Jin
Abstract:
Learning broad world knowledge directly from raw visual data is a fundamental capability of intelligence. We introduce UniVR, the first investigation into simultaneously learning complex reasoning, fine‑grained physical dynamics, and long‑term planning from pure visual demonstrations. At its core, UniVR features VR‑GRPO, a reinforcement learning paradigm with complementary global and step‑level rewards. This approach enforces logical coherence and physical consistency throughout the reasoning process without requiring task‑specific heuristics or image‑text pairs. To train and evaluate UniVR, we construct VR‑X, a large‑scale benchmark curated from 16 diverse sources spanning long‑horizon manipulation, spatial puzzles, and physical reasoning. It is the first comprehensive suite to assess these heterogeneous capabilities under a purely visual protocol. Remarkably, UniVR achieves up to a 25% improvement on VR‑X, and its superior visual reasoning also boosts performance on various multimodal understanding benchmarks. These findings underscore the vast potential of reasoning within visual spaces, with all code, data, and models are open‑sourced for further research.
Authors:Tapan Parikh
Abstract:
When a language model must pick one answer from a large space of equally valid options, which does it pick ‑‑ and how often is it the same answer every other model picks? Asked to "pick a word ‑‑ any word," 44 models chose "serendipity" 41% of the time. We characterize this convergence with a deliberately minimal instrument: 31 single‑turn prompts, each naming a category with many valid one‑word answers ("Name a tree."), asked four times per model with no system prompt. Analysis is exact‑match on normalized tokens ‑‑ no embeddings, no judge ‑‑ at about a dollar per model. That models converge is well documented; our contribution is the instrument itself ‑‑ the One‑Word Census ‑‑ and what it reveals about the structure of the convergence. We score each model by answer‑choice surprisal: the average ‑\log2 probability of its answers under the pooled answers of all other models, leave‑one‑out. Convergence is extreme ‑‑ in 7 of 31 categories one answer takes over 80% of all answers ‑‑ yet conformity varies more than fourfold across models, and the variation is structured. Persona‑ and community‑tuned models are the most divergent; the newest mainline flagships are the most conformist, producing almost no answer no other model gave. Within four lineages (Claude, GPT, Qwen, Grok) conformity rises with each generation ‑‑ but reverses for the latest flagship Claude and GPT models, a possible early signal of repositioning at the top tier. Rankings are robust to roster composition (leave‑one‑family‑out rho = 0.985). Against human category‑production norms, the field is more concentrated than people in 18 of 20 shared categories. All prompts, transcripts, and code are public.
Authors:Kaiwen Zheng, Junchen Fu, Wenhao Deng, Hu Han, Joemon M. Jose, Xuri Ge
Abstract:
Recent advances in multimodal large language models (MLLMs) have significantly improved the performance of multimodal emotion recognition (MER) and enabled interpretable description generation by jointly modeling video, audio, and language, etc. However, these performance improvements are often accompanied by an increase in model parameter size (e.g, at least 7B), which simultaneously incurs high computational costs and reduces inference efficiency, thereby hindering real‑time deployment on resource‑constrained platforms such as robots and mobile devices. This raises a fundamental question: do we really need the multimodal MER model larger than 1B parameters for high‑quality MER? In this paper, we challenge the assumption that larger models are inherently necessary and proposes a lightweight MER framework (called Light‑MER), which achieves better and faster multimodal sentiment understanding and recognition through knowledge distillation. It can transfer knowledge from a strong, large‑scale teacher model to a lightweight sub‑billion‑parameter student model, aiming to preserve rich multimodal emotion reasoning and recognition while substantially improving deployment efficiency. Specifically, we introduce two new optimization strategies to enhance knowledge transfer: (1) a new optimal transport loss that combines Sliced Wasserstein Distance with hidden‑state alignment, and (2) a new multi‑reward optimization strategy based on GRPO that balances MER performance and efficiency, aimed at further enhancing the learning capabilities of student models. Extensive experiments on nine benchmark datasets demonstrate that Light‑MER achieves state‑of‑the‑art performance while significantly improving inference efficiency. This highlights the strong potential of small multimodal emotion language models for future research. Code is available at https://github.com/GAIR‑Lab/Light‑MER.
Authors:Mingzhen Xu, Haonan Guo, Di Wang, Yinghua Qu, Zhiliang Zhou, Lei Zhang, Huiwen Yao, Rui Zhao, Fengxiang Wang, Gang Wan, Bo Du, Liangpei Zhang
Abstract:
Hyperspectral foundation models learn transferable spectral‑spatial representations from large‑scale unlabeled data. They provide an effective paradigm for adapting to downstream hyperspectral image (HSI) classification tasks with limited labeled samples. However, spectral band configurations vary substantially across sensors, which makes direct model transfer difficult. Existing adaptation strategies often compress, select, or reshape the original spectra to match model‑specific input requirements. These operations may discard useful spectral information and weaken local spectral continuity. To address this problem, we propose MBTI, a Multi‑Branch efficient fine‑tuning framework for Hyperspectral Image classification. MBTI adapts hyperspectral foundation models to downstream classification tasks while preserving full‑band spectral information. First, we introduce a spectral‑continuity‑preserving multi‑branch preprocessing strategy. The original HSI is divided into multiple continuous spectral subsets, and a band reuse mechanism is used when the remaining bands cannot form a complete branch. This avoids invalid padding and unnecessary spectral loss. Second, independent Low‑Rank Adaptation (LoRA) modules are inserted into each branch. They enable different spectral intervals to learn task‑specific discriminative features while keeping most pre‑trained parameters frozen. Finally, a multi‑branch channel attention fusion module adaptively recalibrates and integrates features from all spectral branches. Experiments on three public hyperspectral datasets show that MBTI achieves competitive and superior performance compared with representative classification methods. Under the final rank‑8 configuration, only about 2.33%‑‑2.36% of the parameters are trainable. The code will be available at https://github.com/Azhenmiddleblock/MBTI/tree/main.
Authors:Shuwei Huang, Tianyao Luo, Jicheng Liu, Daizong Liu, Pan Zhou
Abstract:
Image super‑resolution (ISR) has witnessed remarkable progress with diffusion models and flow matching. The dominant text‑to‑image (T2I) based approaches leverage large‑scale foundation models as generative priors, achieving impressive perceptual quality but at the cost of massive model sizes and prohibitive training expenses. Recent flow‑matching‑based vision‑only approaches have made significant strides; however, they adopt standard flow formulations that transport from a pure Gaussian prior to the data distribution, discarding the rich structural information already present in the low‑quality (LQ) input. Furthermore, existing single‑step acceleration techniques often forfeit the model's multi‑step inference capability. In this paper, we propose Residual Flow Matching for Image Super‑Resolution (RFMSR), a vision‑only framework that centers the source distribution at the LQ latent, reducing transport distance and preserving structural priors throughout the flow trajectory. We further introduce a two‑phase training strategy: Phase I pretrains the velocity field via conditional flow matching, while Phase II applies end‑to‑end supervision to the single‑step prediction while retaining the velocity loss across all timesteps, achieving high‑quality single‑step generation without sacrificing multi‑step refinement. Extensive experiments demonstrate that RFMSR achieves comparable or even superior perceptual quality compared to state‑of‑the‑art (SOTA) methods. The source code is available at https://github.com/Faze‑Hsw/RFMSR.
Authors:Ruikang Li, Molin Li, Jiarui Wu, Zhe Wei, Pengpeng Liu, Tianfan Xue
Abstract:
When a real‑world scene is captured by a smartphone camera and viewed on its screen, the displayed image often differs noticeably from the original scene in color, brightness, and contrast. This gap persists despite substantial advances in both modern cameras and displays. A key reason is that most pipelines factor the high‑dimensional capture‑to‑display process into two separately calibrated camera and display stages, and then connect them through low‑dimensional color transforms, leading to information bottlenecks and inevitable error accumulation. To address this systemic challenge, we propose Color Pass‑Through, an end‑to‑end learned framework that operates directly on captured images. Our key insight is to treat the camera and display as a coupled system rather than calibrating them in isolation. Coupling the camera and display yields two practical advantages: (1) it brings the entire real‑world scenes to the display via end‑to‑end optimization, and (2) it allows efficient one‑step calibration for each distinct observer via complete capture‑to‑display path. We validate Color Pass‑Through using both digital and human observers. Compared with representative baselines, our method achieves an average gain of +2.0 points on a 5‑point user study and more than 2x improvement on quantitative metrics, demonstrating improved reproduction of the perceived color of the original scene.
Authors:William Franz Lamberti
Abstract:
Generated tokens are a direct driver of the cost, latency, and energy of generative AI (GAI) code editing. We show the format of feedback is a lever on all three. We compare two deliveries of the same requested changes: a holistic prompt (control) versus the structured, line‑anchored export of FileMark (treatment). FileMark is a VSCodium extension for inline comments on any file. In a paired experiment line anchoring cut generated tokens by 22% (Claude Opus) and 58% (Claude Sonnet), reaching 24%‑80% on files of 100 lines or more, with four of seven models generating significantly fewer tokens after multiple‑testing correction. Correctness rose where models had headroom: +2.0 points pooled and +5 to +7 points for three of five local models. An exploratory experiment in which the harness, not the GAI model, applies function‑level patches shows the correctness benefit grows further when the edit‑application burden is lifted: local‑model correctness on 100+ line files roughly triples under anchoring. Line‑anchored feedback reduces what stronger models spend and improves what weaker models get right.
Authors:Alaa Almouradi, Erchan Aptoula
Abstract:
Multi‑label classification assigns several co‑occurring labels to each aerial scene, yet deployed models often encounter data distributions different from their training. Feature‑statistics augmentation such as MixStyle, EFDMix, and correlated style uncertainty improves generalization at low cost but perturbs channel statistics globally, treating each image as a single style; one class can then contaminate the augmentation of another. Domain generalization is understudied for multi‑label remote sensing; no prior method or multi‑source benchmark targets it. A label‑decoupled augmentation framework is therefore proposed, confining style perturbation to label‑specific regions. Per‑label attention, obtained from a learnable module or from gradient class‑activation maps, yields per‑label feature statistics; these statistics are mixed with cross‑domain samples that share present labels, under independent per‑label coefficients, and features are recomposed by attention‑weighted normalization. Three operators combined with two attention sources produce six variants, evaluated on a leave‑one‑domain‑out benchmark from multi‑label UCM, AID, and DFC15 over six shared labels. Averaged over three splits and five seeds, the best variant attains 71.5% mean average precision, exceeding empirical risk minimization by 5.0 points and the strongest global‑statistics baseline by 1.3 points, with the largest gain on the hardest transfer (up to 7.7 points). Ablations indicate that spatial attention and refreshed localization maps are most influential. The framework adds at most 0.35% parameters, leaves inference unchanged, and appears to offer a generic, inexpensive upgrade path for multi‑label statistics‑based domain generalization. Code is available upon acceptance at https://github.com/Alaa‑Almouradi/Style‑Augmentation‑Upgrade.
Authors:Jiahang Wang, Yirong Yang, Yanqing Zhu, Minghua Luo, Shichao Xie, Fei Liu, Mu Xu
Abstract:
Existing vision‑language navigation methods often couple a VLM with waypoint decoders to produce multi‑step action plans, but they typically lack an explicit closed‑loop mechanism for tracking semantic progress, diagnosing execution failures, and recovering from error accumulation in long‑horizon navigation. To address this gap, we propose ReflectVLN, an agentic VLN framework that organizes decision‑making through bidirectionally interactive intention and execution agents. The intention agent performs subtask decomposition and reflection, generating executable subtask descriptions as corrective plans. Conditioned on these descriptions, the execution agent grounds them into short‑horizon actions under current observations while monitoring sub‑goal progress and detecting off‑track behavior. Crucially, ReflectVLN enables closed‑loop bidirectional communication: the execution agent emits progress and deviation signals to trigger reflection and subtask updates on demand, and the intention agent returns structured guidance that reconditions subsequent actions for recovery. To encourage temporally coherent decisions with interpretable intermediate rationales, we introduce Action Chain‑of‑Thought (Action‑CoT), a path‑conditioned dual‑query training scheme for action generation. Experiments on standard VLN benchmarks show that ReflectVLN improves success rates and path efficiency under a constrained data budget, with favorable training cost and fewer high‑level intention calls at inference time, while providing interpretable intermediate decisions for analysis and collaboration. Code is available at: https://github.com/AIprogrammer/ReflectVLN
Authors:Zebin Yang, Qi Wang, Yunhe Wang, Xiurui Guo, Bo Yu, Shaoshan Liu, Jiafeng Xu, Hao Dong, Meng Li
Abstract:
Vision‑Language‑Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low‑power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception‑execution misalignment and long reaction time. In this paper, we propose Jetson‑PI, a method for efficient VLA deployment on onboard devices via Foresight‑Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representation conditioned on committed actions, enabling the action expert to directly predict actions from the future time step. To reduce reaction time, we introduce confidence‑based scheduling optimization that adaptively balances VLM and action expert invocations, complemented by system‑level accelerations including CUDA graph reuse, GPU‑resident intermediate buffering, and flow unrolling. Extensive experiments demonstrate that Jetson‑PI achieves 8.66x and 5.41x improvements in control frequency compared with naive PyTorch and vla.cpp on NVIDIA Jetson Orin, while outperforming VLASH by 14.8% in average success rate on the LIBERO benchmark. The code of our asynchronous algorithm is available on https://github.com/PKU‑SEC‑Lab/Jetson‑PI, and our efficient llama.cpp‑based inference engine is available on https://github.com/PKU‑SEC‑Lab/Jetson‑PI‑Edge.
Authors:Jiho Hong, Eunae Kang, Sanghyun Kim, Young-Sik Shin
Abstract:
Visual Language Navigation (VLN) aims to enable an embodied agent to navigate complex environments by following natural language instructions. Recent approaches build semantic spatial maps and leverage Large Language Models (LLMs) for reasoning and decision making. Despite these advances, existing systems lack instance‑level object detail and robustness to diverse user queries, limiting reliable navigation in complex indoor environments. To address these limitations, we propose Instance‑Enriched Semantic Maps, a unified framework with three key contributions: (1) Instance‑level two‑and‑a‑half‑dimensional (2.5D) rich information mapping that constructs maps from color and depth observations via open‑vocabulary panoptic segmentation, preserving vertical distinctions and capturing small objects, while storing diverse semantic attributes and natural language captions enriched with room‑level context. (2) Robust query processing via LLM‑based target selection, which dynamically routes queries across type‑specialized experts and integrates their outputs through score‑level fusion, enabling consistent goal selection across diverse query formulations. (3) Storage‑efficient semantic representation that achieves approximately 96% reduction compared to three‑dimensional (3D) scene‑graph approaches while preserving sufficient spatial information for navigation. The proposed 2.5D representation outperforms the 3D baseline by over 27% in prediction‑normalized Area Under the Curve (AUC). In navigation experiments, our method achieves over 17% improvement in object retrieval and over 23% in navigation success compared to the baseline across diverse query types. The project page is available at https://rcilab.github.io/iesm_vln.
Authors:Junhui Wang, Hangtao Zhang, Zhirun Zheng, Li Zeng, Jiejun Xiao, Xi Luo, Lihua Yin, Saiqin Long
Abstract:
Large language models (LLMs) are increasingly deployed as purpose‑specific agents to handle domain‑specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but also purpose‑specific restrictions tailored to their designated roles. Such additional restrictions enlarge the attack surface, particularly to prompt injection (PI) attacks. To defend against such attacks, existing detection methods primarily rely on analyzing input‑output patterns, yet yield limited effectiveness. To address this limitation, we turn to analyzing the hidden activation space and discover that LLMs inherently retain latent policy‑violation (PV) concepts when prompted with requests beyond their designated purpose. Particularly, PV concepts capture the semantics of conflicts between user queries and predefined restrictions, implicitly reflecting LLMs' intrinsic awareness of recognizing policy violations. Building on this insight, we propose PVDetector, a training‑free framework that detects PI attacks during LLM inference by measuring hidden‑state alignment with PV concepts, which are derived offline from the contrastive pairs of policy‑violating and policy‑compliant prompts. Experiments across multiple LLMs and datasets show that PVDetector achieves <1% false negative rate with minimal auxiliary overhead, consistently outperforming state‑of‑the‑art methods. Our code is available at https://github.com/Claresigle/PVDetector .
Authors:Dimitrios Kafetzis
Abstract:
Dual‑core MCUs are asymmetric: on NXP's MCXN947, the second Cortex‑M33 has no FPU, DSP extension, TrustZone, or MPU. We treat the asymmetry as a design input in the Phase 3 dual‑core architecture of SynapticOS, an open‑source Zephyr‑based runtime: the AI runtime (models, NPU/DSP, scheduler) lives on the capable core, and the application core reaches inference only via a message‑based OS service ‑‑ a remote system call. The transport is a pair of lock‑free single‑producer/single‑consumer rings in shared SRAM: one writer per index, free‑running 32‑bit counters, ordering by data‑memory barriers alone (the platform has no cross‑core atomics). Because ring state is shared, a rebooting application core rejoins unaided. Requests carry priority classes, errors and timeouts propagate to the caller, and tensors stage zero‑copy in a shared slot ‑‑ a 27 KB frame cannot exist twice in 64 KB of RAM. Measured on the FRDM‑MCXN947 (both cores 150 MHz): the application core boots in 1,514 us and completes the handshake in 2,554 us, bit‑identical over 11 boots; round trips are 15 us typical / 81 us worst‑case (50 us budget); pushes cost 25 cycles; a 1,913‑serve two‑model soak had zero errors (stub‑NPU latencies bracket transport, not silicon). An MPU region on the runtime core guards the application core's RAM (fault‑injection verified); protection is one‑directional ‑‑ the application core has no MPU, and ARMv8‑M cannot block privileged reads. Two hardware‑revealed defects are reported: releasing the second core into erased flash wedges the whole chip and its debug port (now prevented by a ROM‑API blank check), and a Zephyr flash‑driver Kconfig silently disarmed the devicetree MPU guard (now programmed at runtime). Firmware is 98.9 KB flash (runtime core) and 32.4 KB (application core, 42.6 of 64 KB RAM); 108 tests in 13 suites pass 100%. Apache 2.0: https://github.com/Dimitrios‑Kafetzis/SynapticOS
Authors:Shuchan Wang
Abstract:
Continuous‑time generative frameworks construct probability paths between base and target domains by optimizing time‑dependent velocity fields. While theoretical targets favor straight trajectories, empirical networks develop complex path deformations. This paper presents the Finite‑Time Spectral Sensitivity (FTSS) g(t), a gradient‑free, forward‑pass metric that exposes flow geometry by tracking the root‑mean‑square singular value of the state‑transition matrix. Serving as a continuous proxy for stable rank, g(t) reveals a distinct geometric pathology under data scarcity: while generalizing models maintain stable effective dimensions, overfitting causes a spectral collapse. We leverage this structural phenomenon to develop an internal geometric audit based on g(t). Our framework detects generative memorization using purely internal trajectory dynamics, removing the need for external membership queries or baseline data comparison.
Authors:Dimitrios Kafetzis
Abstract:
Microcontroller runtimes treat the inference pipeline ‑‑ pre‑processing, accelerator invocation, post‑processing ‑‑ as application code: every project re‑implements stage sequencing, buffer sizing, and completion signalling around a library call. We argue these are operating‑system concerns and present the Phase 2 inference engine of SynapticOS, an open‑source Zephyr‑based runtime that makes the pipeline a first‑class OS object. A pipeline is drawn from a static pool, validated against a canonical stage order, and executed by a priority job scheduler (realtime > normal > best‑effort, FIFO per class) with cancellation and a bounded job table; no heap on the inference path. Stage buffers are sized exactly from configuration and tensor geometry for the nine built‑in processors (bounded 4x fallback for user stages); all intermediates live in an ephemeral arena reset per frame, so streaming footprint is constant. We evaluate on the NXP FRDM‑MCXN947 (Cortex‑M33, 150 MHz) and the qemu_cortex_m3 CI target, both running a deterministic stub NPU kernel: engine‑overhead baselines, not silicon throughput. On the board the scheduler adds 92 us over the Phase 1 direct‑HAL bracket (1,130 vs 1,038 us; dispatch 1 us); a 30‑frame, six‑stage face‑detection pipeline averages 4.63 ms/frame (215.8 FPS, stub model included) vs 31.1 ms under QEMU soft‑float, at a constant 2,784‑byte arena peak returning to zero each frame. The PowerQuad DSP is routed and self‑calibrated for FFT and Q15 matmul; end‑to‑end speedups are 5.51x (256‑point FFT) and 1.66x (16x16 matmul), short of the plan's 10x target ‑‑ reported as missed, not re‑scoped. Stage‑boundary profiling now runs live on the board, closing a Phase 1 gap. The engine adds 3.8 KB flash on QEMU and 20.7 KB on FRDM. 99 tests across 13 ZTEST suites pass 100% under emulation. Released under Apache 2.0 at https://github.com/Dimitrios‑Kafetzis/SynapticOS
Authors:Dimitrios Kafetzis
Abstract:
Microcontrollers with on‑die neural processing units (NPUs) have become mainstream, but the system software hosting them has not: production combinations of Zephyr or FreeRTOS with TensorFlow Lite Micro treat AI inference as an application‑layer library, leaving memory fragmentation, accelerator‑state hygiene, and model‑lifecycle guards as recurring application‑developer concerns. We present the Phase 1 foundation of SynapticOS, an open‑source runtime built on Zephyr that treats inference as a first‑class workload. It contributes four cooperating subsystems: (1) a tensor‑aware bump allocator with 16‑byte DMA‑aligned persistent and ephemeral lifetimes sharing a single arena, achieving constant‑time allocation (~154 cycles per call, ~78,000 allocations per second at 150 MHz, invariant across tensor sizes) with zero fragmentation by construction; (2) a four‑state hardware abstraction layer for the NPU and DSP, implemented by a deterministic software stub (for CI under QEMU) and a Neutron‑flavoured backend (for the NXP MCXN947); (3) a three‑state model lifecycle registry with duplicate‑name detection, idempotent load/unload, and hot‑swap guards; and (4) a four‑mark cycle‑accurate profiler. We evaluate on the NXP FRDM‑MCXN947 (dual Cortex‑M33 at 150 MHz) and the qemu_cortex_m3 emulator. Build footprints are 67 KB flash / 184 KB SRAM on FRDM (shell, 128 KB arena) and 24 KB flash / 28 KB SRAM on QEMU (no shell, 8 KB arena). End‑to‑end inference brackets through the deterministic stub kernel measure 1,038 us on FRDM and 781 us on QEMU for a 16x16x3 INT8 input; these are baseline overhead numbers, not Neutron silicon measurements, which arrive with the real SDK invoke path in Phase 2. A 61‑test suite across 10 ZTEST suites passes 100% in 6.6 s on the CI emulator path. SynapticOS is released under Apache 2.0 at https://github.com/Dimitrios‑Kafetzis/SynapticOS
Authors:Kwang-Hyun Uhm, Inhwa Son, Sung-Jea Ko
Abstract:
Automatic voxel‑level grounding of free‑text findings in 3D chest Computed Tomography (CT) is critical for clinical interpretability. However, this task remains highly challenging due to the intricate spatial complexity of large 3D volumes and the heterogeneity of free‑text findings. Existing end‑to‑end approaches often struggle to simultaneously learn the localized feature representations required for accurate 3D segmentation and the complex semantic understanding needed for text alignment, leading to suboptimal grounding performance. To overcome this fundamental limitation, we propose a novel decoupled framework that disentangles the problem into two specialized stages: (1) class‑agnostic lesion segmentation and (2) text‑volume reasoning. This structural separation allows the model to first extract candidate sub‑volumes by localizing potential abnormalities. Subsequently, intensive cross‑modal reasoning is performed to align these localized sub‑volumes with free‑text medical findings. To resolve the spatial ambiguities inherent in local regions, the reasoning module is augmented with explicit anatomical guidance, utilizing relative spatial coordinates and lung lobe priors. Evaluated on the ReXGroundingCT benchmark, our method achieves state‑of‑the‑art performance in overall grounding quality on the official leaderboard. These results demonstrate that decoupling detection from reasoning is a highly effective paradigm for handling the complexity of 3D medical visual grounding. Our code is publicly available at https://github.com/khuhm/DAGG.
Authors:Li Hu, Guangyuan Wang, Peng Zhang, Bang Zhang
Abstract:
We present WanToFight, a generative game engine that simulates real‑time, two‑player The King of Fighters '97 (KOF~'97) gameplay from keyboard input. Prior generative game engines target either single‑player first‑person settings or non‑real‑time cooperative scenarios; multi‑player control, real‑time inference, complex physical interaction, and adversarial gameplay have not been jointly addressed. WanToFight closes this gap with three components built on the Wan‑1.3B video diffusion transformer: a streaming autoregressive generator with block‑causal attention and a rolling KV cache; a visually grounded Player Association module that binds each player's keyboard signal to a character identity; and a gated, locally causal keyboard injection module trained with a single‑player‑to‑full‑gameplay curriculum. A four‑step DMD‑distilled student paired with a pruned VAE decoder sustains 30FPS at 512x384 on a single NVIDIA RTX 5090 over the duration of a complete match. To our knowledge, WanToFight is the first generative game engine to combine multi‑player control, real‑time inference, complex physical interaction, and adversarial gameplay in one system.
Authors:Ruocong Tang, Yang Huang, Xing Fang, Chenyi Yan, Chuike Sun, Jing Wang
Abstract:
Post‑click conversion rate (CVR) is a crucial element in online recommendation systems, which addresses significant challenges such as data sparsity (DS), sample selection bias (SSB), and delayed feedback. However, the impact of item discount rate‑a key factor influencing both pricing and user purchasing behavior, has received limited attention. In this paper, we introduce the Discount‑Aware Network (DANet) to model the relationship between item discount rates and CVR. DANet comprises three main components: 1) a time‑frequency transformation module that utilizes Fourier transform to derive the frequency spectrum and capture the long‑term discount rate trends of items; 2) a distribution de‑bias module designed to mitigate the biases in user‑specific discount rates caused by various purchase combinations and promotional activities, as well as periodic deviations linked to different promotion periods on e‑commerce platforms; and 3) a supervised regression auxiliary task that establishes the explicit item discount labels to enhance the model's performance in terms of value accuracy, facilitating an effective representation of item discount rates. Experimental results on real datasets demonstrate the superiority of DANet, with offline AUC improving by 1.61%, and online A/B test also shows that DANet achieves impressive gains of 3.63% on pCVR and 2.23% on GMV. DANet has been successfully deployed on Alibaba Tmall APP. The code is available at https://github.com/tangrc/DANet.
Authors:Thuc Huynh, Tuan Le, Doanh C. Bui
Abstract:
Weakly supervised whole‑slide image (WSI) classification is widely used in computational pathology because slide‑level labels are easier to obtain than dense region annotations. Existing multiple instance learning (MIL) methods often aggregate large bags of patch embeddings using mainly visual cues, which can retain many non‑informative patches and provide weak alignment between instance features and class‑level disease semantics. We propose Concept‑Guided Pruning and Representation Learning (CGRL), a simple framework that introduces class‑level concept prototypes derived from disease prompts into the MIL pipeline. First, concept‑relevance pruning ranks patch instances by their similarity to class concepts and retains the top‑K concept‑relevant patches for downstream MIL aggregation. Second, concept‑guided contrastive representation learning constructs class‑wise positive and negative patch sets from the same similarity matrix and optimizes target‑class, symmetric auxiliary, and cross‑class separation objectives, thereby regularizing the projected concept space. We evaluate CGRL on TCGA‑BRCA and TCGA‑NSCLC using multiple representative MIL methods. Experimental results show that CGRL improves several model‑dataset combinations, with gains depending on the downstream MIL model and dataset. It achieves particularly clear improvements in accuracy and macro‑F1 while reducing computational cost through concept‑relevance pruning. These findings demonstrate that class‑level semantic concepts provide an effective and practical prior for patch selection and representation learning in weakly supervised computational pathology.
Authors:Chengjie Wang, Jingzheng Wu, Xiang Ling, Tianyue Luo, Chen Zhao
Abstract:
Docker is widely used to create reproducible build environments, but Dockerfile drift, the divergence between a Dockerfile and its evolving source code, can cause CI/CD builds to fail. Existing rule‑based and retrieval‑based repair approaches analyze Dockerfiles in isolation and therefore struggle with context‑dependent failures. We present Cadre, a context‑aware framework for automated Dockerfile drift repair. Cadre uses static analysis to construct a Context‑aware Dependency Graph (CDG), which maps each Dockerfile instruction to its file‑level and inter‑instruction dependencies. Guided by the CDG, Cadre first selects the context causally relevant to a failure and then generates a targeted patch from that context. We also introduce DodeX, a pipeline that mines real‑world Dockerfile drift instances from GitHub Actions CI logs while preserving the complete build configurations omitted by static‑snapshot datasets. Using DodeX, we construct D^3, a benchmark of 1,040 drift instances reproducible locally with the original CI parameters. Across D^3, Cadre achieves a 35.22% repair rate, 2.78× that of the rule‑based baseline and 1.24× that of the best LLM‑based baseline. Its two‑step workflow keeps 95.25% of prompts below 30k tokens and avoids the prompt‑overflow failures that prevent competing LLM‑based methods from producing patches in 41 to 58 cases per method. Ablation results confirm that both the CDG and the two‑step workflow improve repair performance. Cadre's advantage over diff‑only approaches also increases as drift ages across commits, supporting explicit dependency modeling for context‑aware infrastructure‑as‑code maintenance. Code and data are available at https://github.com/dw763j/Cadre.
Authors:Francesco Taioli, Daniel Coelho, Iaroslav Melekhov, Roberto Alcover-Couso, Jose Miguel Grande Saiz, Virginia Fernandez Arguedas, Artur Bekasov
Abstract:
Despite remarkable progress in text‑guided image editing, generative models frequently fail to preserve visual object consistency, defined as the preservation of a subject's key attributes throughout the editing process. We address this limitation through three contributions. First, we introduce ABO‑Edit, a dataset specifically designed to study object consistency, comprising over 12,000 triplets of source images, editing prompts, and high‑quality target images rendered from artist‑designed 3D assets, with multi‑view coverage and human‑verified quality control. Second, we uncover an overlooked property of image‑editing rectified flow models: the conditioning embedding space, not directly supervised during training, encodes a prediction of the final generated image even at high noise levels. Third, exploiting this finding, we propose FlowMirror, a parameter‑free auxiliary loss that supervises this conditioning embedding space. Without architectural changes, our method improves generation quality across several metrics over baselines.
Authors:Yiming Liu, Wenqi Lou, Zhiguang Wang, Zhiwei Ke, Fengrui Zuo, Chao Wang, Xuehai Zhou
Abstract:
Vision Transformers (ViTs) achieve strong accuracy but incur high inference latency. Semi‑structured N:M sparsity can reduce arithmetic cost, yet its theoretical savings often fail to translate into proportional end‑to‑end speedups on modern GPUs. This mismatch arises because deployment latency depends not only on arithmetic reduction but also on execution regularity and hardware scheduling under sparsity. Achieving practical acceleration, therefore, requires coordinated design across sparse execution and sparsity configuration. To this end, we propose a hardware‑software co‑design framework for N:M sparse ViT inference. On the hardware side, we design MD‑SpMM, an N:M sparse CUDA kernel that reorganizes sparse GEMM into micro‑dense, Tensor‑Core‑aligned dataflow and uses inference‑aware adaptive parallelism to sustain utilization. On the software side, we perform layer‑wise sparsity search under explicit end‑to‑end latency budgets using a three‑stage heuristic search with constraint relaxation to avoid premature convergence and enable deployment‑aware sparsity allocation. Experiments on multiple ViT/Swin models and GPU platforms show that the framework achieves over 2.2x latency speedup while maintaining comparable accuracy and delivering superior accuracy under the same latency constraint. The source code is publicly available at https://github.com/liuganhuo/realizable‑nm‑sparse‑transformer.
Authors:Yuhang Yan, Linchao Mou, Bokang Yang, Qingyu Li
Abstract:
Beyond perception, reasoning is essential in remote sensing for advanced interpretation, inference, and decision‑making. Recent advances in large language models (LLMs) have enabled tool‑augmented agents that leverage external tools to perform complex analytical tasks. However, existing studies in remote sensing primarily focus on perception‑oriented tasks, leaving cognitive geospatial reasoning largely underexplored. To address this gap, we introduce TerraLogic, a benchmark for geospatial reasoning. TerraLogic comprises 545 scenario‑driven, hierarchy‑aware tasks, such as hazard vulnerability assessment, urban heat island analysis, and forest fragmentation dynamics, spanning optical, Synthetic Aperture Radar (SAR), and infrared (IR) imagery. It advances evaluation beyond recognition and monitoring toward cognitive‑level geospatial analysis. To facilitate evaluation on TerraLogic, we further propose HieraPlan, a tool‑augmented agent that organizes toolkits into functional hierarchies and performs fault‑tolerant reasoning. HieraPlan enables structured abstraction, robust recovery from tool failures, and stable long‑horizon planning. Extensive experiments demonstrate that current approaches struggle with hierarchical geospatial reasoning, while HieraPlan provides a strong baseline with improved reasoning, cross‑modal generalization, and error handling. The dataset and agent code are publicly available at https://github.com/Ireliya/TerraLogic.
Authors:Zhishan Zou, Guoyan Sun, Zhiwei Wei, Jiancheng Pan, Yujie Li, Mugen Peng, Wenjia Xu
Abstract:
Autonomous UAV systems increasingly rely on multimodal large language models (MLLMs) to operate in complex real‑world environments. Such embodied scenarios require not only understanding the surrounding space but also maintaining a coherent representation of the agent itself. However, existing UAV‑oriented approaches and benchmarks remain largely environment‑centric, primarily focusing on spatial understanding tasks, with the agent's self‑awareness remaining implicit. To address this gap, we introduce SIS‑Bench, a benchmark for evaluating embodied spatial intelligence in UAV scenarios under a unified self‑in‑space formulation. SIS‑Bench organizes evaluation along two complementary dimensions, space and self, and a three‑level hierarchy of perception, memory, and reasoning. It contains 4,856 question‑‑answer pairs across 13 tasks derived from 1,646 real‑world UAV videos through a task‑conditioned construction pipeline with expert verification.Extensive evaluations reveal that current MLLMs exhibit fundamental limitations in modeling dynamic and agent‑centered processes. In particular, we observe a clear imbalance between spatial cognition and self‑awareness, as well as a progressive performance degradation across cognitive levels.Motivated by these findings, we further explore a motion‑aware representation that incorporates self‑related dynamics through optical flow and visual feature fusion. Experimental results show that modeling agent motion consistently improves perception and memory performance, not only in spatial cognition but also in self‑awareness, and generalizes to downstream UAV decision‑making tasks.Our results highlight the importance of self‑awareness for advancing embodied spatial intelligence, and provide both a new benchmark and empirical evidence for motion‑aware self‑in‑space modeling.
Authors:Qihang Zhang, Siyao Zhang, Letao Kang, Wenzhe Liang, Miao Zhang, Zhao Zhang
Abstract:
Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each individual node and all nodes across the traffic network. However, the underlying mechanism by which such global information is modeled and extracted remains insufficiently investigated. Whether global information must be extracted by high‑degree‑of‑freedom adaptive attention or can be captured by a simple global aggregation operator remains unclear. For this purpose, we design a controlled ablation framework that replaces only the spatial mixing module to test attention‑based global interaction. Across six traffic benchmarks, uniform full‑range mixing and standard spatial attention each achieve lower MAE on three datasets, with only a 0.14% difference in mean MAE, while the former reduces node‑scale spatial mixing complexity from O(N2) to O(N). Mechanism analysis further decomposes spatial attention into a row‑uniform global background and a non‑uniform residual. The residual shows dataset‑dependent marginal value, suggesting that spatial attention should be justified by stable gains beyond a row‑uniform global background. The corresponding source code is publicly available at: https://github.com/uuesti/U‑Trans
Authors:Timing Yang, Jinrui Yang, Xinlong Li, Yuhan Wang, Haoran Li, Yanqing Liu, Guoyizhe Wei, Jixuan Ying, Chen Wei, Rama Chellappa, Yuyin Zhou, Cihang Xie, Alan Yuille, Feng Wang
Abstract:
This work introduces a unified formulation for vision models, where diverse forms of visual information beyond natural images, such as masks, depth maps, and other structured visual signals, are all represented as RGB images, while general visual tasks can be converted into a common RGB‑to‑RGB image editing problem. In this paradigm, different types of visual information internally share the same encoding and decoding architecture and parameters as natural images, enabling a single model to transfer across tasks through a unified visual interface, in a way analogous to how language models operate over text. We refer to this formulation as RGB In and RGB Out (RINO). Built upon a generic image editing backbone without task‑specific fine‑tuning, RINO demonstrates robust and competitive zero‑shot performance on both dense understanding tasks such as segmentation and depth estimation (where we unify outputs as RGB), and dense‑conditioned generation tasks such as pose‑to‑image generation (where we unify inputs as RGB). We hope this study provides useful insights toward general unified vision‑language systems, where diverse visual tasks can be expressed, interpreted, and solved through a shared visual language. Code is available at https://github.com/yangtiming/RINO.
Authors:Yusong Li, Pingchuan Ma, Ming Gui, Vincent Tao Hu, Björn Ommer
Abstract:
Learning representations that separate content and style is crucial for controllable generation and compositional generalization. However, diffusion and flow‑based models trained primarily with generative objectives often produce entangled or misaligned factors. To address this gap, we introduce Contrastive Augmented Flow Matching (CAtFM), a framework that integrates contrastive regularization into an invertible flow matching formulation to promote structured content‑style representations. Rather than constraining intermediate latents or velocity fields, we apply contrastive supervision to predicted endpoints during training, enforcing semantic consistency across transported distributions while allowing disentanglement to emerge implicitly, without assuming strictly pure or fully factorized content and style representations. Our main experiments operate in the CLIP embedding space, with additional validation using frozen DINO and ALIGN encoders. Across synthetic data, in‑domain styles, and real‑world benchmarks (ImageNet, WikiArt, DomainNet, and DTD), CAtFM improves content and style retrieval, enhances embedding cluster separation, and achieves stronger open‑set robustness compared to generative and discriminative baselines. Overall, CAtFM provides a simple way to couple discriminative constraints with deterministic transport, improving disentanglement and robustness under distribution shift.
Authors:Saiyue Lyu, Zhitian Zhang, Ruizhi Deng, Thibaut Durand
Abstract:
We present a diffusion based model for asynchronous time series prediction, where the goal is to predict the next inter event time and event type. To address the inherent uncertainty of future events, we introduce ReDiTT, a retrieval augmented conditional diffusion transformer that operates in latent space. ReDiTT retrieves structurally similar latent sequences from a memory bank during both training and inference and incorporates them as reference conditions through cross attention. This retrieval based conditioning allows the model to attend to relevant temporal dynamics and provides global structural guidance for generation. As a result, ReDiTT stabilizes long horizon forecasting and improves sample diversity. Experiments on seven real world datasets demonstrate state of the art performance on next event prediction and long horizon forecasting. Our code is available at https://github.com/BorealisAI/ReDiTT.
Authors:Yuxuan Ren, Fan Yang, Jianhua Yao, Yatao Bian
Abstract:
Small molecules, crystals, and proteins all reduce to atoms in 3D space, yet their generative pipelines remain fragmented across domains, each with its Small molecules, crystals, and proteins all reduce to atoms in 3D space, yet their generative pipelines remain fragmented across domains, each with its own graph, equivariant, or frame‑based architecture. Cross‑domain training would mitigate per‑domain data scarcity, but direct generation in 3D coordinate space cannot easily handle the heterogeneous structural priors of all three domains, and no prior latent autoencoder is simultaneously lossless and architecturally general across all three. We introduce SinAE, a single‑architecture flow‑matching autoencoder for molecules, crystals, and proteins, with vanilla Transformer encoder and decoder and no equivariant, graph, or domain‑specific operators. Rather than requiring the encoder to capture fine‑grained geometry, SinAE shifts the reconstruction burden into an iterative flow‑matching decoder, achieving near‑lossless reconstruction across domains and reducing reconstruction errors by orders of magnitude relative to prior latent baselines. The same per‑token latent supports a standard Diffusion Transformer prior that reaches strong performance on molecular, crystal, and protein generation benchmarks. Joint molecule‑‑crystal training strictly improves both domains, providing direct evidence of cross‑domain transfer through a shared atomic latent. Code is available at https://github.com/BlueWhaleLab/SinAE .
Authors:Jinjian Wu, Jiaqi Tang, Wei Wei, Yingying Yan, Jianmin Chen, Botong Geng, Lei Zhang, Qifeng Chen
Abstract:
Image Quality Assessment (IQA) in open‑world environments remains challenging due to limited generalization and interpretability. Recent approaches based on multimodal large language models (MLLMs) introduce textual reasoning for quality prediction, yet their judgments rely heavily on semantically biased internal representations, making them insensitive to low‑level perceptual degradations. We propose IQA‑T1, a tool‑based visual evidence reasoning framework that augments MLLM reasoning with explicit perceptual observations. During inference, the model autonomously invokes specialized analysis tools to generate structured visual evidence, such as noise residual maps, gradient statistics, and frequency spectra, which are progressively integrated into the reasoning process. To support this paradigm, we construct Q‑Tool, a dataset containing 11k multimodal reasoning chains grounded in tool‑generated evidence. Extensive experiments on seven IQA benchmarks show that IQA‑T1 achieves the best overall performance across datasets while producing interpretable and evidence‑grounded quality assessments. Code and dataset are available at https://github.com/zibuyu‑02/IQA‑T1.
Authors:Sukriti Tiwari, BHVSP Subrahmanyam, Nidhi Goyal, Sai Amrit Patnaik
Abstract:
EEG‑to‑image evaluation should distinguish visual fidelity from recoverable meaning. Yet EEG‑derived reconstructions are blurry, distorted, and low‑detail, causing SSIM, LPIPS, and CLIP to penalize semantically recoverable outputs or reward plausible but incorrect ones. We analyze 6,855 ground‑truth/reconstruction pairs from ATM, ENIGMA, BrainVis, and DreamDiffusion using semantic probes, caption harshness and blind‑spot rates, and controlled degradations. Pixel metrics show near‑zero correlation with semantic consistency, while representation metrics conflate perceptual and semantic errors. We therefore introduce a BCI‑aware framework in which four VLMs assess image pairs through structured questions, producing Tolerant Perceptual Alignment Scores (T‑PAS) and Tolerant Semantic Alignment Scores (T‑SAS). Their consensus is distilled into the BCI‑Coherence Score (BCS), a compact evaluator achieving a T‑PAS MAE of 0.079 (r = 0.700) and a T‑SAS MAE of 0.082 (r = 0.850) on our data. Human validation shows highly reliable joint coherence judgments, with Cohen's kappa = 0.882 +/‑ 0.174 and Krippendorff's alpha = 0.882, supporting perceptual‑semantic recoverability over generic visual similarity. Code and resources are available at https://sukt03.github.io/BCS/.
Authors:Seung-gyeom Kim, Areum Kim, Yongjae Yoo, Sukmin Yun
Abstract:
Recent 4D Gaussian Splatting (4DGS) methods often fail under fast motion with large inter‑frame displacements, where Gaussian attributes are poorly learned during training, and fast‑moving objects are often lost from the reconstruction. In this work, we introduce Spatiotemporal Position Implicit Network for 4DGS, coined SPIN‑4DGS, which learns Gaussian attributes from explicitly collected spatiotemporal positions rather than modeling temporal displacements, thereby enabling more faithful splatting under fast motions with large inter‑frame displacements. To avoid the heavy memory overhead of explicitly optimizing attributes across all spatiotemporal positions, we instead predict them with a lightweight feed‑forward network trained under a rasterization‑based reconstruction loss. Consequently, SPIN‑4DGS learns shared representations across Gaussians, effectively capturing spatiotemporal consistency and enabling stable high‑quality Gaussian splatting even under challenging motions. Across extensive experiments, SPIN‑4DGS consistently achieves higher fidelity under large displacements, with clear improvements in PSNR and SSIM on challenging sports scenes from the CMU Panoptic dataset. For example, SPIN‑4DGS notably outperforms the strongest baseline, D3DGS, by achieving +1.83 higher PSNR on the Basketball scene.
Authors:Kazushi Kato, Koji Inoue, Taiga Mori, Divesh Lala, Tatsuya Kawahara
Abstract:
In human dialogue, we achieve smooth communication by expressing nonverbal cues such as eye contact, nodding, and facial expressions with precise timing. It is expected for conversational avatars to express these cues appropriately to realize natural and human‑like interactions. This study focuses on nodding, which is crucial for demonstrating active listening and encouraging further user utterances. We propose a model that predicts both timing and kinematic parameters representing the motion features of listener nodding in real time. The proposed model consists of a timing prediction module and a kinematic parameter prediction module. Each implements a dyadic attention network over the speaker and listener channels based on the technique of Voice Activity Projection (VAP). Unlike conventional models, this approach enables real‑time prediction of kinematic parameters based on the specific context of the dialogue rather than just predicting the timing. Furthermore, we demonstrate the effectiveness of fine‑tuning the kinematic parameter prediction module initialized from the trained timing prediction module. The proposed model is lightweight and capable of real‑time operation, and it has been integrated into an avatar dialogue system. Subjective evaluation experiments shows that our proposed method significantly outperforms both a baseline with stochastic timing and another with fixed‑motion nodding. The code and trained models are available at https://github.com/MaAI‑Kyoto/MaAI.
Authors:Jixiang Luo
Abstract:
Autonomous research systems are often evaluated as one‑shot paper generators: given a topic, they produce a manuscript and a small set of experiment logs. This framing hides the operational problem that makes such systems difficult to trust: research is long‑running, branching, failure‑prone, and dependent on auditable handoffs between agents and humans. XScientist is a git‑like research protocol and operating system for this setting. It orchestrates idea generation, experiment execution, manuscript drafting, self‑review, repair, quality gating, daemon scheduling, and reproducibility artifacts as one continuously observable pipeline. The central design choice is to treat each run as a portable research artifact rather than only as a PDF. XScientist exports an Agent‑Native Research Artifact (ARA), a protocol that records an exploration DAG, per‑node code and outputs, claim‑to‑evidence anchors, content hashes, provenance, and re‑execution hooks. This makes each generated paper inspectable as a science exploration tree: failed branches, repaired experiments, ablations, and manuscript claims remain connected to the nodes that produced them. The system also includes deterministic integrity forensics, sample gates, truth contracts, reviewer‑oriented repair loops, and long‑running daemon controls. This paper describes the current XScientist architecture, the ARA protocol surface, and the practical safeguards needed to move autonomous science from single‑run demos toward reproducible, reviewable, and forkable research infrastructure. The implementation and manuscript source are maintained in the public GitHub repository at https://github.com/smileformylove/XScientist.
Authors:Shipeng Liu, Zhanping Song, Liang Zhao, Dengfeng Chen
Abstract:
Crack segmentation is essential for infrastructure inspection and structural health assessment, but existing high‑performance methods typically require task‑specific pixel‑level annotations and training. Text‑promptable vision foundation models enable zero‑shot deployment, yet their final mask proposals are poorly suited to thin, fragmented, and low‑contrast cracks, whose evidence may be suppressed, truncated, or over‑expanded during mask generation. We find that language‑conditioned semantic responses within the SAM3 decoder preserve more continuous and complete crack evidence than its final masks. Based on this observation, we propose Semantic‑Edge Response Decoding (SERD), which interprets internal responses as a dense crack‑likelihood field, calibrates them with a lightweight edge prior, and generates crack masks using a unified global threshold, without annotation or fine‑tuning. Experiments on six public datasets show that SERD consistently improves over native SAM3 and outperforms the compared zero‑shot and open‑vocabulary segmentation methods, achieving an average Crack IoU of 61.14%, 4.63 points higher than SAM3. Further analyses show that most gains arise from directly decoding internal semantic responses, while edge calibration improves structural recovery and false‑positive control without increasing end‑to‑end inference overhead. These results suggest that, for thin and non‑compact targets, internal continuous responses can provide a more transferable interface than the final masks of foundation models. Code is available at: https://github.com/xauat‑liushipeng/SERD
Authors:Yike Wang, Huaisheng Zhu, Zhengyu Hu, Yige Yuan, Zhengyu Chen, Shakti Senthil, Hannaneh Hajishirzi, Yulia Tsvetkov, Pradeep Dasigi, Teng Xiao
Abstract:
We revisit the evaluation of automatic harness evolution for LLM agents. Existing harness evolution methods use unit test cases to search for harness configurations and then report final performance on the same public benchmark. This protocol raises two fundamental concerns. First, harness evolution is itself an iterative search procedure that repeatedly evaluates and revises candidate harnesses using task feedback. As in agentic test‑time scaling, it should therefore be compared with simple task‑level search baselines under matched feedback and inference budgets to determine whether its gains arise from improved harness design or from additional search alone. Second, because the search and the final evaluation share the same benchmark, the reported gains risk overfitting to that specific task set. To address these concerns, we conduct an extensive evaluation comparing harness evolution with simple test‑time scaling and discovery baselines under comparable feedback and inference budgets, and also evaluate evolved harnesses on held‑out tasks to assess whether the discovered improvements generalize. Experiments on Terminal‑Bench 2.1 with GPT‑5.4 and Claude Opus 4.6 show that automatic harness evolution does not consistently outperform simple test‑time scaling methods and exhibits limited generalization. Our results raise important questions about the effectiveness of automatic harness evolution and highlight the need for fairer evaluation protocols and benchmarks for automatic harness design. Our code is available at https://github.com/rethinking‑harness‑evolution.
Authors:Andre Hora, José Miguel Rojas, Romain Robbes
Abstract:
Software systems have unique testing characteristics. Some projects can emphasize unit tests, while others may focus on end‑to‑end testing. Test organization may vary across ecosystems: in languages like Python and Java, tests are typically placed in dedicated folders, whereas Go and Rust projects commonly co‑locate tests with source code. These distinctions make it harder to understand how a project approaches testing. In this paper, we present TestMiner, a tool for exploring software testing in GitHub repositories. TestMiner provides an overview of a project's testing practices, including test statistics, test location, test metrics across releases, and dependencies related to testing. We used TestMiner in an undergraduate Software Testing course, where 50 students explored the testing practices of real‑world GitHub repositories. Overall, students expressed positive feedback regarding TestMiner. They were able to critically explore a variety of testing practices, including test organization, test evolution, test fixtures, mocking, and edge‑case testing. TestMiner is available at: https://andrehora.github.io/testminer. Screencast: https://youtu.be/w1sBgLTq‑7Y.
Authors:Jiahao Luo, Hao Zhang, Jianqi Chen, Yijie He, Jiaxu Zou, Michael Vasilkovsky, Sergei Korolev, Sergey Tulyakov, Chaoyang Wang, Peter Wonka, James Davis, Jian Wang
Abstract:
We present RegHead, a framework for constructing semantic blendshape sets for animatable non‑humanoid head avatars. With a fixed expression vocabulary, semantic blendshapes provide a low‑dimensional and interpretable animation interface and support cross‑identity retargeting. Building such blendshape sets remains expensive because (i) expression‑consistent supervision is scarce, (ii) generated 4D assets typically lack correspondence, and (iii) facial motion is highly localized. We propose (1) a large‑scale dataset of non‑humanoid identities paired with a shared expression vocabulary, obtained by expanding a small artist‑rigged library via fine‑tuned image editing; (2) a dense stochastic anchor motion representation tailored to localized facial deformations; and (3) a fast feed‑forward registration model that converts unregistered expression meshes into a corresponded blendshape basis by predicting anchor‑based deformations from the neutral shape. Experiments show that our approach produces higher‑fidelity expression meshes than baselines, while running orders of magnitude faster than optimization. We further demonstrate real‑time retargeting from human face tracking signals to non‑humanoid characters, capturing both head pose and localized facial motions. Our project page is available at https://snap‑research.github.io/RegHead/.
Authors:Vishwajith Ramesh
Abstract:
Attention can be viewed as an online learner over context, yet existing test‑time memories cannot certify that dropping a token leaves outputs unchanged or delete its influence outright. We introduce Support Vector Attention (SV‑Attention), a max‑margin memory whose weights are support coefficients of a one‑class SVM with fixed box parameter C. Its active‑set partition gives reserve tokens exactly zero weight, certifying output‑preserving eviction; a reversible incremental solver deletes a token to recover the state produced by retraining without it under the same C. In fp64 experiments, decrement and refit recover identical partitions whenever the optimum is unique, and their decision functions match to a median deviation of about 10^‑9 (10^‑13 on learned keys); the 10^‑2 worst case is confined to ill‑conditioned duplicates and remains below coefficient decay in every regime. The exact path reuses the maintained KKT inverse in a custom backward. Training uses a separate stabilized batched approximation and does not carry the exact‑deletion certificate; it reaches 9,125 tokens/s on a 3.22M‑parameter model, while remaining 35.8 times slower than an MPS softmax reference. At matched budgets, certified selection reaches 0.86 vs. 0.32 rare‑item recall and retains 0.80 vs. 0.05 deterioration hours on real MIMIC‑IV streams. We also demonstrate surgical forgetting, exact editing, patient‑record deletion, and a forgettable retrieval memory over real sentence embeddings. On enwik8, the hybrid obtains 2.178 BPC vs. 2.383 for a matched‑state sliding‑window Transformer across seven seeds (8.6% paired improvement, p=0.001); a three‑seed TinyStories result is directionally positive but not significant (p=0.057).
Authors:Mohamed Abdessalem Bal
Abstract:
Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predominantly on correlational recovery metrics ‑‑ cosine similarity between ground‑truth directions and decoder atoms. We show this conflates two distinct claims: decoder‑geometry alignment and encoder‑activation behavior. We reproduce the superposition phase diagram of Elhage et al. (2022), identifying a convergence artifact at high sparsity and an under‑described diffuse sharing regime at extreme overcompleteness. We reproduce the TopK‑versus‑L1 comparison of Gao et al. (2024), with direct evidence of L1 shrinkage. Our central result is causal: subjecting every recovered feature to ablation and steering, we find up to 77% of features passing a recovery bar (cosine >= 0.90) in a degraded SAE ‑‑ and 9% in a well‑trained one ‑‑ are causally inert: the matched atom never fires when the feature is present, including matches at cosine ~1.000. We package the method as sae‑causal‑audit, a model‑agnostic instrument with a deterministic pipeline. Re‑auditing refines the finding: inertness decomposes by cause into structural inertness (antipodal‑pair geometry, present in good SAEs) and competitive inertness (a TopK pathology of degraded SAEs), and by direction into read‑ and write‑inertness, which five antipodal pairs dissociate completely ‑‑ unmonitorable yet steerable through the same atom, with steering specificities of 143‑310 attached to zero ablation effects. We document why byte‑exact reproducibility is unavailable by construction, and propose reporting it as a stack of claims with explicit scopes. Applying the instrument to a production SAE reproduces the pattern at small scale (14% inert) and surfaces an atom‑collision signal: a handful of atoms recur as the nearest match for dozens of unrelated concepts, replicated across three batches.
Authors:Luis Loo, Ulisses Braga-Neto
Abstract:
We present an agentic approach to autonomous neural operator discovery based on an AI scientific community, which consists of a swarm of virtual laboratories that interact under a citation‑based economy of influence. Highly‑cited labs found new labs that follow their research direction and replace non‑performing labs. Each virtual lab contains three agents: an LLM planner that proposes an architecture, a numerical worker that trains and measures it, and an LLM reviewer that participates in cross‑lab peer review. All labs share a common vocabulary consisting of DeepONet (branch‑trunk), Fourier, Transformer (attention), wavelet, and residual convolutional neural operator building blocks. We evaluate the neural operator AI scientific community on five problems, namely piecewise regression, the linear advection and Burgers 1D PDEs, and the Navier‑Stokes and Darcy flow 2D PDEs, while repeating the simulation three times for each problem. The results show that the neural operator AI scientific community is capable of discovering high‑accuracy, low‑parameter‑count neural operator architectures. All 9,623 LLM calls are logged and audited, which reveals that the virtual lab LLM planners choose to hybridize in 99.8% of their logged decisions, consistently returning multi‑family hybrids. Moreover, we conducted an ablation study by replacing the LLM agents in each lab by rule‑based alternatives, which caused the scientific community to collapse to non‑hybridized single‑family stacks in several cases, showing that LLM agency is needed to preserve diversity. The results suggest a no‑free‑lunch theorem for neural operators: there is no universal winner. The code, configurations, and the complete LLM transcripts are released at https://github.com/luislootx/AI‑SC.
Authors:Vincent Giap, Eric Wang, Cris Nguyen
Abstract:
Published molecular docking scores depend on the receptor, ligand, software, search box, seed, and preparation choices; a paper reporting only the score has published a number with unknowable provenance. We ask whether such claims can be re‑executed from their own published records. We introduce MERS‑Dock, a 16‑field Minimum Executable Reporting Set, and a deterministic E0‑E4 executability ladder over audited field states. In 236 open‑access SARS‑CoV‑2 main‑protease docking papers, only 8.1% met the essential‑field rule for direct re‑execution (E3), 47.9% were blocked by a missing foundational field (E1), and none reached E4; mean field completeness was 49.1% and the search‑box centre was reported by only 33.9%. We validated the audit against two independent human reviewers on a 65‑paper stratified sample: inter‑reviewer agreement was 92% (pooled Cohen kappa 0.87), and the automated agent matched humans on the execution‑blocking fields while over‑calling two non‑blocking fields; the resulting E‑class was 68% concordant with humans and, where it differed, human review lowered the executable count ‑‑ so the low‑executability finding is confirmed, not inflated. Reporting did not improve over 2021‑2026 (completeness vs year Spearman rho = ‑0.01). A bounded within‑paper re‑execution shows the reproduction gap is a box‑coverage geometry effect, not box‑size disclosure. We read E‑class as an executability gate, not a reproducibility predictor, and release Mpro‑DockExec as a traceable measurement layer for digital‑library and evidence‑synthesis systems deciding what is checkable in published computational claims.
Authors:Yuvraj Sehgal, Sneh Patel, Mahsa Panahandeh, Naser Ezzati-Jivan, Francois Tetreault
Abstract:
Machine learning models for system diagnostics rely on kernel execution traces to capture fine‑grained system behavior, but collecting production traces in industrial systems is costly due to runtime overhead, storage demands, and privacy constraints. We present TraceSynth, a diffusion‑based framework for generating synthetic kernel traces that augment limited real data for downstream ML tasks. TraceSynth models traces as multi‑channel sequences (event types, timestamps, CPU affinity, thread identifiers, and process metadata) using a Transformer‑based denoising diffusion process with constraint‑guided repair to enforce system invariants. Across six benchmarks, results show strong workload dependence. For deterministic, compute‑heavy workloads (scimark2), synthetic augmentation achieves 87.2% F1‑Macro at context length L=4096, only 2.6 percentage points below real‑only baselines. Context length is the dominant quality factor, with L=4096 yielding a +104% relative improvement over L=256, while constraint‑guided repair improves synthetic data quality by up to 4.3%. Ablation studies show that lightweight 2‑channel models retain 97‑99% of the performance of full 6‑channel models at roughly half the computational cost. TraceSynth supports cost‑effective augmentation of kernel execution traces in production observability pipelines and helps identify when synthetic data can substitute for limited real traces.
Authors:Robel Mamo, Rajitha de Silva, Grzegorz Cielniak, Taeyeong Choi
Abstract:
While visual navigation has been extensively studied in agricultural robotics, most existing systems assume daytime conditions. In fact, deploying autonomous robots at night offers significant advantages, including 24‑hour crop and soil monitoring, fruit harvesting, and nocturnal pest detection. Modern vision‑based systems, however, rely heavily on large‑scale well‑annotated image datasets, which remains challenging to obtain for nighttime operation scenarios. To address this, we propose an unsupervised image translation framework that converts daytime plant‑row RGB images into near‑infrared (NIR) nighttime counterparts without requiring pixel‑to‑pixel supervision. This enables the direct reuse of daytime semantic labels for training nighttime perception models. In particular, by incorporating a pre‑trained Contrastive Language‑Image Pre‑training (CLIP) model, the proposed framework is designed to preserve semantic consistency during day‑to‑night translation. Additionally, a visibility mask is introduced to account for the limited effective range of NIR illumination in nighttime scenes. We conduct comparative evaluations with state‑of‑the‑art image translation baselines and demonstrate higher image qualities, as supported by improved performance in downstream semantic segmentation for nighttime visual navigation. For evaluation, we utilize AgriNight‑‑a novel dataset comprising 428 daytime and 549 nighttime images collected using night‑vision‑equipped mobile robots in agricultural fields and manually annotated with pixel‑wise semantic labels‑‑and introduce it as the first benchmark for nighttime agricultural visual navigation. We also perform real‑time autonomous navigation experiments with a physical robot operating at night. The data and code are available at: https://github.com/mamorobel/AgriNight.
Authors:Arastoo Zibaeirad, Marco Vieira, Thomas Zimmermann
Abstract:
Given a vulnerability‑fixing commit, trigger localization asks which specific statement turns the vulnerable program state into a concrete unsafe operation. This question is harder than binary vulnerability detection because the answer demands interprocedural, causal reasoning: in a substantial fraction of real‑world CVEs the triggering statement lies several call layers outside the patched function, beyond the reach of static rule sets and pattern‑matching language models alike. We present AutoTrace, an agentic pipeline that localizes vulnerability triggers by exploring a code property graph layer by layer, with LLM agents deciding where to look next and deterministic admissibility gates deciding what evidence is required before a trigger can be reported. Agents never accept a trigger on their own authority; every reported trigger is backed by explicit evidence drawn from the graph, so the pipeline covers both intra‑ and interprocedural vulnerabilities without relying on ungrounded model judgment. On the full InterPVD benchmark, AutoTrace reaches 75.0% VulnHit and 80.8% FuncHit, surpassing the prior state of the art on the same corpus. Building on the same machinery, we construct SinkTrace‑Bench, a dataset that exposes each vulnerability as a source‑to‑sink (S2S) causal chain from attacker‑controlled input through propagation to the dangerous operation, drawn from matched vulnerable and patched program states. It comprises 1,542 verifier‑confirmed, perfectly balanced vulnerable/safe samples whose label fidelity we audit against expert annotations. Benchmarking frontier LLMs on it, we find that even the strongest struggle to separate the matched pairs, exposing the causal‑reasoning gap that trigger localization targets. Artifact available at https://github.com/Erroristotle/AutoTrace.
Authors:Vinicius Anjos de Almeida, Nícolas Henrique Borges, Leonardo Vicenzi, Helena Kociolek, Sarah Miriã de Castro Rocha, Frederico Nassif Gomes, Júlia Cristina Ferreira Ribeiro, Lucas Emanuel Silva e Oliveira
Abstract:
Large language models (LLMs) are increasingly being explored for clinical decision support, but their reliability in complex oncology treatment planning remains unclear. We evaluated agentic LLM systems for breast cancer treatment recommendation generation using 72 real clinical cases across stages I to IV and 1,147 case‑specific rubrics generated through Asymmetric Information Rubric Generation (AIRG), in which the rubric generator had access to real clinical decisions unavailable to the evaluated models. Seven pipelines were compared, including single‑LLM baselines, tool‑augmented systems, and multi‑agent architectures with fact checking and autonomous subagent spawning. The best‑performing configuration, Claude Opus 4.8 with the D&C+SA pipeline, achieved a global score of 0.594 \pm 0.025. Tool use and increased agent autonomy had mixed effects, improving performance in some settings but degrading it in others. Performance varied by clinical domain and disease stage, and oncologist‑led error analysis revealed persistent clinically relevant failures, including incorrect or missing recommendations, flawed justifications, citation errors, outdated claims, and overconfidence. These findings suggest that agentic LLM systems can generate clinically relevant breast cancer recommendations, but remain insufficient for unsupervised clinical use.
Authors:Yufei Cai, Xuesong Niu, Hao Lu, Kun Gai, Kai Wu, Guosheng Lin
Abstract:
Current visual generation models are capable of producing high‑quality content, yet they lack a coherent perception of the spatial structure. Existing generative novel view synthesis methods typically introduce explicit geometry priors, which enforce spatial consistency but inherently restrict generalization in large view changes. In contrast, recent interactive generative methods favor implicit scene modeling, offering greater flexibility at the cost of precise camera control and geometry consistency. In this paper, we propose MetaView, a diffusion‑based monocular novel view synthesis framework that enables rendering under large view changes from a single image. Our key insight is to combine implicit geometry modeling with minimal yet essential explicit 3D cues: we incorporate implicit geometry priors from a feed‑forward geometry perception network to regularize structure without imposing restrictive reconstruction pipelines, while leveraging metric depth to anchor the generation to a metric scale. This design allows MetaView to achieve both geometry consistency and precise controllability. Extensive experiments demonstrate that, under challenging monocular large viewpoint changes, MetaView significantly outperforms existing methods and exhibits superior generalization. Our code is publicly available at https://github.com/KlingAIResearch/MetaView.
Authors:Hang Yuan, Chen Li, Wenjun Ma, Tadahiko Murata, Yuncheng Jiang
Abstract:
Precision molecular design aims to discover personalized drug candidates through joint control of multiple conditions, such as biological relevance and molecular design strategies. Biological relevance reflects cellular functional states under disease or perturbation conditions, while molecular design strategies provide complementary guidance in terms of structural intentions and property optimization. In this study, we propose JoPMol, a jointly controlled precision molecular generative model that integrates biological states encoded by gene expression profiles with molecular structure information expressed in text, and chemical properties quantified by numerical values within a unified modeling framework. This formulation enables coordinated generation and optimization of candidate molecules under joint condition control. Experimental results show that JoPMol outperforms state‑of‑the‑art methods across multiple evaluation metrics. Moreover, JoPMol demonstrates strong generalization ability in both transfer tasks and biologically grounded simulation scenarios, validating its effectiveness for precision molecular design. The source code is publicly available at https://github.com/hala‑yh/JoPMol.
Authors:Elaheh Hassani, Durga Mandarapu, Qi Yu, Hanghang Tong, Ariful Azad
Abstract:
Network alignment identifies node correspondences across different networks and is a fundamental primitive in many data science applications, including social network analysis, fraud detection, and knowledge graph integration. However, state‑of‑the‑art network alignment methods often achieve high accuracy by repeatedly constructing and updating dense matrices, sacrificing scalability in the process. To address this scalability limitation without compromising alignment accuracy, we present FastAlign, a scalable, sparsity‑aware framework for optimal transport‑based network alignment. Rather than introducing a new alignment model, FastAlign preserves the original OT formulation and reinterprets its computation as a set of recurring mixed sparse‑dense operations. FastAlign combines sparsity‑aware graph computation with domain‑specific kernel fusion, including a custom SpMM kernel. Our results show that FastAlign achieves alignment quality comparable to state‑of‑the‑art OT‑based methods while substantially reducing end‑to‑end runtime up to 3.89x‑9.45x on CPU and 2.24x‑32.54x on GPU.
Authors:Gengyu Zhang, Haiyin Ran, Zhengbao He, Yuhang Liu, Hanling Tian, Zhehao Huang, Xiaolin Huang
Abstract:
As the scale of large pre‑trained models continues to grow, fine‑tuning them under limited memory budgets has become increasingly challenging. Low‑Rank Adaptation (LoRA), currently one of the most widely adopted parameter‑efficient fine‑tuning (PEFT) methods, mitigates this challenge by optimizing only low‑rank adaptation matrices, thereby greatly reducing the number of trainable parameters. With the parameter overhead substantially reduced, the activations retained for backpropagation have emerged as the primary remaining memory bottleneck during LoRA fine‑tuning. To address this, we propose CARE‑LoRA, a data‑aware Compressed Activation REconstruction framework. By exploiting the inherent projection structure of LoRA, CARE‑LoRA replaces the full input activation with the low‑rank compressed activation naturally produced by the LoRA branch. It further computes a lightweight reconstruction matrix during the forward pass with negligible additional computation cost, which is used during backpropagation to reconstruct the gradient signal, thereby keeping LoRA matrices fully trainable. Extensive experiments across diverse models and downstream tasks demonstrate that, while substantially reducing the overall memory footprint, CARE‑LoRA achieves competitive or even superior performance compared with standard LoRA and representative LoRA variants. Our code is publicly available at https://github.com/fishandyu/CARE‑LoRA .
Authors:Neerav Gupta
Abstract:
Plasma diagnostic models for tokamak fusion devices are almost universally evaluated on clean, complete sensor data. In practice, fusion diagnostics fail regularly: acquisition systems start late, individual sensors die, and signal dropouts cluster precisely when a plasma disruption is approaching. We present the first systematic robustness benchmark for plasma diagnostic ML using the TokaMark dataset of 11,573 MAST shots, evaluating XGBoost, LSTM, Transformer, and the TokaMark CNN baseline across six physically‑grounded failure scenarios and three imputation strategies. We introduce the Robustness Score (RS) for standardized cross‑architecture comparison. Our central finding is that disruption‑proximate sensor failure (corruption injected in the final window timesteps) collapses sequence model performance (LSTM +212% NRMSE) while a statistical feature model remains comparatively stable (XGBoost +37%). Forward‑fill imputation eliminates nearly all degradation from random dropout for sequence models (LSTM +57% to ~0%), but offers little help when the end of the window is corrupted. Shot‑level alarm evaluation using ground‑truth disruption timestamps reveals that LSTM alarm detection collapses to TPR=0.00 under proximate sensor failure, while mean‑fill imputation recovers it to TPR=1.00, a reversal of the pattern observed in NRMSE. Plasma current emerges as the single most critical diagnostic across all architectures (+73% to +140% upon removal). Code, data, and trained checkpoints are available at https://github.com/Neerav‑Gupta/tokamark‑robustness.
Authors:Tri-Nhan Vo, Dang Nguyen, Sunil Gupta
Abstract:
Large‑scale text corpora have become a quiet bottleneck in modern NLP, not just in storage, but in the accumulated cost of training, fine‑tuning, and continual learning. We propose a text dataset distillation framework that reduces corpora to as little as 0.1% of their original size while preserving downstream task fidelity. We approach distillation through the lens of influence functions, which quantify each sample's contribution to the downstream objective, a natural and principled basis for selection. We introduce Trajectory‑Aware Knowledge Estimation (TAKE), which convolves the knowledge‑based influence along the training trajectory into a single per‑sample knowledge score, capturing informative samples. These scores serve as sample weights within a discrete Optimal Transport objective, guiding prototype selection from a synthetically generated candidate pool. We evaluate TAKE on downstream accuracy across text classification and natural language inference tasks at extreme compression (0.1% or 20 samples/class), showing that data efficiency is achievable without sacrificing task fidelity. The approach is theoretically grounded, with broader implications for coreset construction and data‑centric AI. We release our source code at https://github.com/votrinhan88/take.
Authors:Zhiyang Dou, John U. Onyemelukwe, Hangxing Zhang, Heng Zhang, Minghao Guo, Yunsheng Tian, Michal Piotr Lipiec, Joshua Jacob, Chao Liu, Peter Yichen Chen, Yuri Ivanov, Wojciech Matusik
Abstract:
Differentiable simulators have advanced policy learning and model‑based control across robotic tasks. Yet actuator dynamics remain underexplored and can be a major source of sim‑to‑real error, particularly on low‑cost platforms, where the linear current‑to‑joint‑torque approximation τ= K_t I becomes unreliable because of friction, hysteresis, backlash, and thermal effects. Accurate actuator models can also support force perception and integrated force/position control. We present NeuralActuator, which jointly predicts (i) a torque surrogate for trajectory propagation on low‑cost servo platforms, (ii) external forces with a contact‑probability gate for sensorless force perception, and (iii) a motor‑condition score for a supervised joint, distinguishing normal from mechanically restricted operation. A twin‑arm teleoperation system records robot states and actuator telemetry alongside external‑force labels, yielding the Neural Actuation Dataset (NAD). The torque‑surrogate head is trained through differentiable simulation from pose trajectories without ground‑truth joint‑torque measurements. A Transformer captures temporal dependencies while enabling real‑time inference. We validate NeuralActuator on a 5‑DoF OpenManipulator‑X, a 6‑DoF SO‑101 from LeRobot, and a 7‑DoF Franka Emika Panda, spanning three actuator families and costs from approximately \500 to more than \30,000. The low‑cost platforms support physically plausible dynamics and force evaluation, while the offline Franka experiment provides a payload‑force‑estimation benchmark. We also demonstrate motor‑condition estimation and improved behavior‑cloning performance using NeuralActuator as a pretrained module. We release the dataset, code, and hardware configurations on the project page: https://frank‑zy‑dou.github.io/projects/NeuralActuator/index.html.
Authors:Claudio Rota, Luca Cogo, Simone Bianco, Raimondo Schettini
Abstract:
Color correction is a key component of camera image signal processing (ISP) pipelines, encompassing illuminant discounting and colorimetric mapping of device‑dependent sensor responses to device‑independent color spaces, such as CIE XYZ. Despite extensive research, accurate color correction remains challenging due to the non‑linear relationship between camera sensor responses and CIE XYZ color space, as well as to the increasing presence of highly chromatic and spectrally complex LED illuminants. We propose a color correction framework based on illuminant‑adaptive three‑dimensional lookup tables (LUTs), which we call Color Correction LUT (C^2LUT). Our method combines a chromaticity‑aware illuminant representation with a non‑linear color transformation, enabling accurate correction under illuminants spanning a wide range of chromaticities and spectral complexities. We employ Tucker tensor decomposition to represent the LUTs, ensuring that computational requirements remain sufficiently low for deployment in camera ISPs. In addition, we introduce a large‑scale illuminants dataset comprising 1,473 spectral power distributions, with different chromaticities and spectral profiles. Experiments across multiple cameras, illuminants, reflectance datasets, and real captured images demonstrate consistent improvements over existing methods for color correction, reducing CIE ΔE_00 by up to 20% and angular error by up to 18% while remaining compatible with modern camera hardware constraints. Code and datasets are available at https://github.com/claudiom4sir/C2LUT.
Authors:Evelyn D'Elia, Weishu Zhan, Giulio Turrisi, Giulio Romualdi, Giuseppe L'Erario, Raffaello Camoriano, Wei Pan, Daniele Pucci
Abstract:
Reinforcement learning (RL) algorithms classically suffer from poor sample efficiency. In robotics, a recent line of work has emerged addressing this problem by encoding physics priors in the learning process. However, most of these approaches are validated on well‑defined, low‑dimensional benchmark systems rather than high‑dimensional robots with complex nonlinear dynamics. In this paper, we introduce SKooP (Symmetric Koopman Predictions), an approach combining the advantages of morphological symmetries with those of a Koopman model learned via autoencoder to enhance policy learning. SKooP learns a Koopman model of the system dynamics alongside the policy. The resulting Koopman predictions are used as privileged observations for the critic, allowing the agent to learn based on smoother, more informative features. We also incorporate group symmetries into the actor, critic, encoder and decoder networks to produce a highly equivariant policy. The SKooP approach is validated via in‑depth analysis of the learned Koopman models and symmetric policies to showcase how each of these influences the agent's performance. We also show that the learned policies are transferable to different simulation environments. Our results show that SKooP consistently reduces convergence time and increases the learned reward for multiple challenging bipedal locomotion tasks on a quadruped robot. Project page: https://evelyd.github.io/SymmetricKoopmanPredictions
Authors:Muxin Liu, Xiaoyang Lyu, Tianhe Ren, Peng Dai, Xiaoshan Wu, Zhiyue Zhang, Jiaqi Zhang, Jiehong Lin, Shaoshuai Shi, Xiaojuan Qi
Abstract:
We present FoundationGeo, a two‑stage framework that explicitly bridges relative and metric prediction via spatial calibration and principled data design. Stage 1 learns a high‑fidelity, affine‑invariant geometry model by initializing with DINOv3 and training on a curated 10.2M‑sample multi‑domain corpus with complementary local‑detail supervision, yielding sharp boundaries and strong cross‑domain generalization. Stage 2 moves beyond global scaling by introducing lightweight pixel‑wise calibration fields for metric estimation: a scale field for spatially varying metric alignment and a ray‑direction correction field that mitigates directional bias in point‑map geometry, together producing metrically consistent 3D point maps. Beyond model design, we identify camera intrinsic coverage, especially focal length distribution mismatch between training and test data, as a key bottleneck for zero‑shot metric generalization: performance drops sharply when test intrinsics fall outside the training distribution. To address this, we synthesize additional training data across diverse focal lengths using a Blender‑based data engine, repairing under‑covered focal regimes and improving robustness under intrinsic shift. Extensive zero‑shot evaluations across seven benchmarks show that FoundationGeo significantly strengthens cross‑domain robustness, staying near the top across diverse domains while avoiding the sharp cross‑domain performance drops observed in other methods. This consistency translates into the best overall performance, surpassing heavier baselines by over 5.2% on average.
Authors:Xiaojian Liu, Han Xu, Jianqiang Xia, Zhixuan Li, Ke Xu, Yiwei Dai, Xinran Chen, Changwo Wu, Yuchen Li
Abstract:
Reinforcement learning with verifiable rewards (RLVR) optimizes LLMs using sparse verifiable final‑answer rewards. This sparse anchor reliably verifies whether a trajectory succeeds but provides no direct feedback on the reasoning path that produced it. Before success, prerequisite progress on hard problems receives no reward signal; after success, outcome rewards cannot distinguish well‑organized correct trajectories from redundant or locally flawed ones. We introduce SCOPE‑RL (Scaffolded Chain Optimization with Process Efficiency), a two‑stage framework that densifies this anchor while retaining the GRPO update: Adaptive Scaffolded RL adds prefix‑decomposed verifiable rewards on answer‑hidden sub‑question chains before success, and Quality‑Aware Process RL applies correctness‑gated process‑shape rewards to refine correct trajectories after success. An expert‑validated Step‑Quality Evaluation Protocol evaluates useful‑step density, error localization, and token efficiency beyond final‑answer accuracy. On Qwen3‑8B‑Instruct trained on DAPO‑Math and Big‑Math, SCOPE‑RL improves average accuracy by up to 11.2 pp and reduces reasoning tokens by up to 27.1% over outcome‑only GRPO; the gains hold under GSPO and on Qwen3‑0.6B‑Instruct, indicating that reward‑signal densification is complementary to policy‑update‑level RLVR advances. Code and data are available at https://github.com/tokencraft‑lab/SCOPE‑RL.
Authors:Zhe Xiao, Longfei Li, Xu He, Haoying Wu, Zixing Zhang, Mingyu Liu
Abstract:
Symbolic expressions can effectively characterize and predict circuit behavior, but deriving them directly from circuit schematics is challenging. This process requires accurate visual‑to‑symbolic construction of circuit structure from images and correct multi‑step symbolic derivation, both of which impose strict correctness requirements. This work proposes AutoVSR, an automated framework for visual‑to‑symbolic generation of circuit expressions using Vision Language Models (VLMs). By reconstructing circuit diagrams into an executable intermediate representation (Executable IR) and leveraging a symbolic solver for reasoning, AutoVSR significantly improves the accuracy of symbolic expression generation. AutoVSR introduces two key innovations: an IR construction method guided by component rule retrieval and verification‑based feedback, and a symbolic solver implemented as a planning agent equipped with a symbolic tool library for reliable multi‑step derivation. Compared with end‑to‑end VLM approaches and specialized methods on the main symbolic expression generation task, AutoVSR achieves accuracy improvements of 30.01‑‑59.45% and 41.96‑‑51.84%, respectively. Moreover, AutoVSR surpasses closed‑source state‑of‑the‑art VLMs in inference cost and computational efficiency. Code is available at https://github.com/LongfeiLi1/AutoVSR.
Authors:Chen Huang, Qi Zheng, Ruiqin Zheng, Long Zeng, Yuantong Xu
Abstract:
Model editing keeps large language models (LLMs) up to date without retraining, but temporal facts expose a limitation of the prevailing locate‑and‑edit paradigm: an update is not always a replacement. When a fact changes, the new answer should become current while the old answer may remain correct in historical time contexts. Building on this insight, we use causal tracing to show that LLMs already support this distinction via a two‑stage internal computation: early MLP layers retrieve a time‑agnostic subject representation, and later layers modulate it with temporal context to yield the time‑correct answer. Motivated by this finding, we introduce PRISM Edit, which optimizes a single polysemous representation across temporal contexts and leverages the model's inherent modulation pathway to route it to temporally correct predictions, without any architectural modification. We evaluate on TimeConflict, a new temporal editing benchmark we introduce, and on temporally augmented CounterFact. PRISM Edit improves over the best baseline by +23.3 Temporal Consistency (TC) and +33.7 Current Relative‑time Score (CRS) on average while being more than 2x faster. Code and data are publicly available at https://github.com/AnonymousStudy972/PRISM‑Edit.
Authors:Tianyu Xiong, Rui Li, Suning Ge, Jiaqi Yang
Abstract:
Reconstructing 3D scenes from unordered images remains bottlenecked by expensive Structure‑from‑Motion (SfM) preprocessing and frozen pose interfaces. We present SalientGS, a unified SfM‑to‑3D Gaussian Splatting (3DGS) pipeline. Its central contribution is importance‑guided Markov Chain Monte Carlo (MCMC) Gaussian allocation, which aggregates multi‑view residuals into per‑Gaussian underfit and redundancy signals. These signals define a smooth importance‑weighted sampling distribution that biases both birth and relocation toward underfit regions. This reallocates capacity from well‑fit areas without altering the underlying stochastic gradient Langevin dynamics (SGLD). SalientGS achieves end‑to‑end reconstruction in 15 minutes with state‑of‑the‑art perceptual quality. The supplementary material provides dedicated sections for Per‑Scene Qualitative Comparisons and Per‑Image Learned Perceptual Image Patch Similarity (LPIPS) Analysis, including failure cases. Code and evaluation scripts are available at https://github.com/Six‑Bit‑TX/SalientGS.
Authors:Suhaas Garre, Emily Ritchie, Sushant Mehta, Edwin Chen
Abstract:
A large share of day‑to‑day work in professional domains happens inside PDF files: benefits packets, leases, datasheets, clinical guidelines, construction plans. Benchmarks for document AI have generally measured the required capabilities in isolation: OCR, layout analysis, chart reasoning, table QA, document VQA. A high score on any one of them does not necessarily reveal whether a model can answer a realistic question that someone in the field would actually ask about a specific PDF. GDP_pdf is a benchmark built to measure this directly. It consists of question‑document pairs authored by working professionals in ten fields, and a candidate question was kept only when at least two frontier multimodal models failed it in a way that mattered: a wrong answer, missed decisive evidence, or a fabricated claim, rather than a superficial difference such as style. Each item comes with a rubric of atomic criteria, so we can report a graded rubric score as well as a strict task‑level pass rate, and each item is tagged against a taxonomy of eleven capabilities in three tiers, spanning text extraction and grounding, table and chart comprehension, cross‑referencing, spatial reasoning, and abstention on unsupported queries. We evaluated seven frontier models on the 100‑item benchmark. The best model passed only 15% of the items and the worst passed 1%. Most errors trace back to a small set of recurring loss patterns: misaligned tables, misread charts, skipped footnotes and exclusions, miscounted floor‑plan symbols, scan noise, and amendments that supersede earlier text.
Authors:Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong
Abstract:
Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution. Online reinforcement learning with verifiable rewards (RLVR) has emerged as a key direction for scaling their capabilities. However, this paradigm is bottlenecked by verifiable data scarcity and online RL inefficiency. To break these barriers, we introduce ScaleCUA, a unified framework that scales online RL for CUAs via verifiable task synthesis and efficient training. At the data level, we design VeriGen, an end‑to‑end framework for generating verifiable RL tasks through iterative docker interactions and a multi‑agent feedback loop. Scaled to 100+ concurrent agent workers via a shared docker interaction probe, this pipeline produces 24K+ verifiable tasks and nearly 3K high‑quality RL tasks. To maximize sample efficiency, we propose Frontier Sampling, which tracks per‑task capability and allocates rollouts to the current learning frontier. On the training side, we further design Visual Context Segmentation, a sliding window over recent visual context that balances rollout and training‑engine pressure, yielding a 2.83x training speedup over step‑wise decomposition. Together, ScaleCUA achieves 68.7% on OSWorld and 54.0% on ScienceBoard, establishing new state‑of‑the‑art performance among open‑source computer use agents. Code, models, and datasets are available at https://github.com/THUDM/SCALE‑CUA.
Authors:Ruilan Gao, Letian Jin, Yu Zhang
Abstract:
SLAM methods based on 3D Gaussian Splatting (3DGS) have demonstrated impressive tracking and mapping performance, but typically require additional geometric information from external depth sensors. Meanwhile, recent SLAM systems that leverage geometric priors from pre‑trained feed‑forward models enable real‑time dense reconstruction, yet often discard original RGB information during optimization, thus degrading overall reconstruction quality. We present GeoGS‑SLAM, an online monocular dense reconstruction system that combines the 3DGS‑based map representation with learned geometric priors. Given uncalibrated RGB input, we first employ a feed‑forward visual geometry model to predict camera and scene priors. The Gaussian scene map is then expanded by directly sampling Gaussian primitives from both RGB input and geometric priors. Camera poses and the scene map are jointly optimized through a coarse‑to‑fine strategy that minimizes both photometric and geometric losses. To ensure global consistency, we further incorporate online loop closure detection and pose graph optimization. Extensive experiments across indoor and outdoor benchmarks demonstrate that GeoGS‑SLAM achieves superior rendering quality and tracking accuracy compared to state‑of‑the‑art methods while maintaining online real‑time performance. Project page: https://rlgao.github.io/geogs_slam.
Authors:Chunzheng Zhu, Lei Tian, Bohan Tan, Ziqi Zhou, Yuxuan Sun, Yijun Wang, Chengchao Lv, Yilin Wen, Yijun He, Jinghao Lin, Yihang Chen, Cheewei Tan, Qianshan Wei, Lei Zhao, Bin Pu, Kenli Li, Yuan Xue, Jianxin Lin
Abstract:
The growing ability of large language models and vision language models to jointly interpret and reason over images and text is reshaping medical agents, moving them from task specific predictors toward autonomous systems that perceive, reason, plan, remember, and act in clinical environments. This work departs from the capability first perspective of existing literature and instead begins from clinical deployment, asking what tasks, contamination resistant benchmarks, and interactive training environments are required before medical agents can be trusted in practice. Medical agents are formalized as sequential decision making systems under partial observability, together with a three level autonomy taxonomy spanning assisted, cooperative, and fully autonomous operation. The field is organized along a unified scaling spine consisting of framework scaling, capability scaling, and environment scaling. Within this framework, clinical environment scaling, the integration of tools, data, and clinical gyms, is identified as the most actionable yet underexplored direction for agents operating in PACS, EHR, and FHIR ecosystems. Clinical self evolution, where agents improve through interaction with their environments rather than parameter scaling alone, is further positioned as a key research frontier, drawing insights from self improving agents, agent gyms, and test time compute scaling. Applications across radiology, pathology, ophthalmology, and hospital workflows are examined together with deployment challenges including hallucination, cascading failures, and fairness. By consolidating more than 300 references, with particular emphasis on advances from 2025 to 2026, this work provides a roadmap toward trustworthy, self improving medical imaging systems for real clinical practice.
Authors:Haojie Huang, Linfeng Zhao, Haotian Liu, Zhang Ye, Si-Yuan Huang, Mingxi Jia, Boce Hu, Fangzhou Lin, Yu Qi, Dian Wang, Robin Walters, Robert Platt
Abstract:
Representing manipulation actions as 2D trajectories in the camera plane provides a compact and interpretable basis for learning complex 3D manipulation policies. However, it also creates challenges from out‑of‑frame trajectories and limited precision. We propose Pix2Act, an imitation learning method that addresses these challenges by generating continuous image‑space keypoint trajectories in each camera plane and losslessly recovering end‑effector poses via triangulation. This reformulates high‑dimensional 3D control as a simpler, more learnable 2D prediction problem. Crucially, it aligns observations and actions in the same coordinate space, enabling equivariant transformations to jointly rotate individual camera images together with their image‑space actions. We analyze the symmetry properties of this augmentation and design a network architecture that can fuse multiple camera views while respecting their per‑view rotations. As a result, Pix2Act implicitly enlarges the support of the data distribution and learns invariant action structures across transformations, yielding improved generalization and overall performance. Across diverse simulated and real‑world manipulation tasks, Pix2Act outperforms state‑of‑the‑art baselines and remains robust under camera perturbations.
Authors:Chenyu Hu, Bang Wang
Abstract:
A thematic corpus is a collection of semantically coherent documents that collectively describe different aspects of a shared thematic event. Such a corpus typically contains hundreds or even thousands of documents. While users' interests in a thematic event often span multiple dimensions, Query‑Focused Summarization (QFS) aims to generate summaries tailored to users' queries. However, existing QFS datasets lack event‑oriented summarization, and most QFS methods struggle with large‑scale corpora. To address these challenges, we propose the Query‑Focused Event Summarization (QFES) task and construct the QFESum dataset, which contains 8 thematic events, 16,684 documents, and 104 queries. Furthermore, we introduce a two‑stage QFES framework consisting of Query‑Focused Retrieval with Adaptive Thresholding (RAT) and Query‑Focused Summarization based on Hierarchical Clustering (SHC). Experimental results on QFESum show that RAT and SHC consistently outperform the baselines, demonstrating their effectiveness for QFES. The dataset and code are publicly available at https://github.com/sarcasm‑hcy02/QFES‑QFESum.
Authors:Yi Ting Shen, Kentaroh Toyoda, Alex Leung
Abstract:
Safety evaluation of large language models (LLMs) relies largely on single‑turn attack datasets and single‑judge scoring, underestimating risk from adaptive multi‑turn adversaries and reporting a single success rate that does not separate partially actionable outputs from those carrying complete operational detail. We propose AMT‑X (Adaptive Multi‑Turn Exploitation), a phase‑structured multi‑turn red‑teaming framework. Unlike prior multi‑turn attacks that rely on ad hoc escalation or free‑form per‑goal plans, AMT‑X casts the attack as an explicit, reproducible multi‑phase state machine driven by semantic signals from the victim, and replaces single‑judge scoring with a multi‑role jury whose phase‑conditioned checklists gate success on actionable harm. Across six frontier victim models (queried under their default safety alignment, without added moderation layers) and seven Moderation sub‑categories, AMT‑X attains overall attack success rates of 97.6‑100% under a lenient score threshold, but 66.7‑78.6% under a stricter gate requiring complete, real, and operational detail: a gap of up to 33 percentage points between partially and fully actionable harm.
Authors:Yue Fang, Zhibang Yang, Fangkai Yang, Xiaoting Qin, Liqun Li, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
Abstract:
Large language model (LLM) agents increasingly rely on external tools served by shared providers and accessed by heterogeneous downstream agents. Existing approaches improve tool use on the agent side through parameter updates, prompt refinement, or agent‑side memory, making tool knowledge difficult to share and limited to behaviors observed in past tasks. We argue that reusable tool knowledge should instead be maintained by the tool provider. We introduce ToolAtlas, a graph‑based framework that builds a persistent provider‑side tool memory of tool capabilities, failure boundaries, and cross‑tool compositions through execution‑verified probing. At inference time, agents query the tool memory via adaptive graph traversal. Across two MCP‑based benchmarks spanning eight services, ToolAtlas outperforms existing tool‑side optimization and agent‑side memory baselines by up to 21.61% in pass@1 and 18.61% in pass@4. The same tool memory also transfers across environment instances and agent frameworks without retraining or task‑time exploration, yielding up to 24.16%/16.22% and 17.49%/14.27% relative gains in pass@1/pass@4, respectively. Ablation studies show that these gains arise from combining tool‑centered memory organization with capability‑guided execution probing. These results establish provider‑side tool memory as an effective and reusable paradigm for tool servers. Our code is in: https://github.com/PuppyKnightUniversity/ToolAtlas.
Authors:Haoyu Gu, Lekai Qian, Haowu Zhou, Qi Liu, Shuai Wang
Abstract:
Music creation is fundamentally a process of revision. Yet symbolic music generation remains dominated by paradigms that produce complete sequences from scratch, with limited support for selective modification. Edit‑based methods have proven effective for text transformation tasks, but remain largely unexplored for symbolic music. We trace this absence to the representational level: conventional event‑based music encodings lack the structural properties required by explicit music editing. In contrast, the BEAT encoding, a beat‑grid‑anchored representation originally designed for autoregressive generation, possesses structural properties amenable to editing. We propose BeatEdit, the first framework for symbolic music generation based on explicit edit operations, recasting generation as producing new content by editing a draft rather than synthesizing from scratch. BeatEdit comprises three complementary mechanisms along an axis of increasing edit density: per‑token sequence tagging for error correction, iterative refinement for accompaniment editing, and tag‑then‑fill for segment completion. All these mechanisms share a single encoding and pre‑trained backbone, achieving higher precision and perceptual quality than autoregressive and diffusion methods across all three tasks, while remaining efficient, with single‑pass inference completing in under 100 ms. Cross‑encoding evaluation further reveals that encoding design substantially influences editing effectiveness, with notable encoding‑method interaction effects. Code is available at https://github.com/Haoyu‑Gu/BeatEdit‑code
Authors:Joyjeet Singh
Abstract:
Deep equilibrium models promise input‑adaptive implicit computation: harder problems should demand more solver iterations, and the solved equilibrium should encode the result of genuine iterative inference. We report a cautionary study of a port‑Hamiltonian DEQ with a learned initialization on two reasoning tasks ‑‑ ProofWriter entailment over frozen DeBERTa embeddings and a BFS‑verified graph‑reachability benchmark ‑‑ in which the implicit computation is a silent no‑op. Across tasks, seeds, and controlled ablation arms, the solved equilibrium equals the solver's start point to numerical precision, and bypassing the solver entirely changes test accuracy by +0.00 percentage points in 18 of 19 training runs. Controlled interventions falsify the tempting explanation: removing the anchoring term reproduces every result, and retraining with noise‑decoupled starts yields a solver that converges to the noisy start while the decoder learns to ignore it. The single escaping run diverges instead (\|h^‑z_0\|=171), producing a co‑adapted noise channel whose removal improves accuracy. Iteration counts are uncorrelated with ground‑truth difficulty (r=0.009), and the full apparatus never outperforms a two‑layer MLP on either task. We trace the mechanism to gradient starvation along two distinct routes, show that the standard zeroing ablation is confounded and gives wildly seed‑dependent answers where the correct substitution test gives a stable zero, and distill a four‑test diagnostic protocol for auditing claimed implicit computation. All experiments run on a single free Colab GPU; code, raw logs, and analysis scripts are released.
Authors:Jeongsoo Kim
Abstract:
Recently, Vision Transformer (ViT)‑based models have exhibited remarkable performance in image super‑resolution. However, the quadratic computational complexity of ViTs with respect to spatial resolution severely constrains their efficiency, leading to high latency and massive memory consumption. To alleviate this, various window‑based attention mechanisms have been proposed; yet, they inherently compromise the long‑range dependency modeling that is the primary advantage of ViTs. To overcome these limitations, we propose the Clustered Unit‑level Similarity Transformer (CUST), a novel architecture that efficiently integrates global and local information. Specifically, CUST enables each patch to aggregate and attend to similar patches within a broadened regional scope outside its local window, thereby capturing extensive contextual understanding. Furthermore, it employs overlapping attention windows to capture local dependencies, while explicitly extracting high‑frequency details by computing the residual difference between the original features and their downsampled‑upsampled counterparts. Comprehensive experiments demonstrate that our proposed model achieves a practical balance between computational efficiency and restoration performance. It achieves a lower memory footprint and faster inference speed compared to recent global context or lightweight models under realistic constraints. Code is available at [https://github.com/jwgdmkj/CUST].
Authors:Sunyoung Jung, Jiwoo Park, Yoonseok Choi, Kyobin Choo, Ming-Hsuan Yang, Seong Jae Hwang
Abstract:
Diffusion Transformers (DiTs) have advanced video generation with high‑quality, temporally coherent results. However, extending them to motion transfer, which requires following reference motion while aligning with a target prompt, remains challenging due to limited understanding of motion and structure representations within DiTs. We analyze video DiTs at the attention‑head level and identify distinct heads specialized for motion and spatial structure. Based on this insight, we propose a head‑aware controllable motion transfer framework that requires no parameter updates. Our method refines motion cues from motion‑specialized heads via semantic correspondence guidance and preserves structure through selective feature injection. This head‑level control not only enables accurate motion transfer but also provides an interpretable foundation for controllable video generation with DiTs.
Authors:Varun Ramesh Jois, Antonella DiLillo, James Storer
Abstract:
Videocalling has become a popular form of communication in the world today, with many companies providing free services for it. However, there are still millions of people around the world that experience poor quality videocalls due to limitations in bandwidth. This despite, most people having the required hardware. In this paper we present a novel framework for enhancing highly compressed videocalls. We show, that with as little as 10 frames of the face, we can rapidly (in under 100 seconds) train a model to enhance that instance of the videocall. The model can be trained either prior to or during the call, enhancing the rest of the call by producing better quality video. The video conferencing application need not be modified ‑ it can be off the shelf with our system as a layer on top that trains quickly then simply lets the video conferencing application (e.g. Zoom) run as usual, where our system intercepts and improves images before they are displayed. The model is designed to run in realtime on low‑compute devices such as a typical laptop CPU. Experimentally, we show that the model significantly improves quality of compressed face video both quantitatively as well as perceptually. Code can be found at https://github.com/varun‑jois/FSFVE.
Authors:Varun Ramesh Jois, Antonella DiLillo, James Storer
Abstract:
There's been a surge in adoption of video conferencing applications for both personal and business use cases. However, the bandwidth limitations faced by many users worldwide may restrict the optimal use of such applications. Although deep learning offers a solution for enhancing low bit rate videos, most models today are either hard to incorporate with modern compression standards or require specialized hardware to run such as significant GPUs making these models impractical. To address these issues, we introduce the Realtime Face Video Enhancement (RTFVE) model which can be easily incorporated with any video decoder and can run in realtime on ordinary CPUs. Experiments show that our model improves perceptual quality over the compressed video baseline at multiple low bitrate settings. The source code will be made available at https://github.com/varun‑jois/RTFVE.
Authors:Yeonseo Lee, Taeyeop Lee, Hyosup Shin, Guebin Hwang, Sungho Jo
Abstract:
Dexterous grasp generation across robot hands is challenging because hands differ in kinematic topology, actuation dimensions, and native command spaces. We introduce GraspGraphNet, a topology‑aware grasp generation framework that represents each hand as a URDF‑derived kinematic graph and directly generates executable palm poses and joint configurations. GraspGraphNet combines hierarchical object surface encoding, differentiable forward kinematics, and dynamic world‑edge message passing to model evolving robot‑object interactions. It applies conditional flow matching directly in executable palm‑pose and joint‑state space, avoiding post‑processing optimization, inverse kinematics, and retargeting. Using a shared model trained on Barrett Hand, Allegro Hand, and Shadow Hand, GraspGraphNet achieves an average success rate of 83.48% with 40ms inference time per grasp on a 40‑object benchmark. Without retraining, the same model achieves 72.70% success on controlled finger‑removal variants, demonstrating robustness to hand‑topology variations. These results suggest that graph‑structured hand representations can effectively support dexterous grasp generation across robot hands with different kinematic structures. Project: https://lysees.github.io/graspgraphnet‑page
Authors:Varun Ramesh Jois, Antonella DiLillo, James Storer
Abstract:
Face super‑resolution is the task of increasing the resolution of an image containing a face thereby adding finer detail. It is a ubiquitous task in many computer vision applications and quite often the user isn't even aware that it is being performed. However, doing it with high fidelity is challenging as it is an ill‑posed problem. In this paper we present a reference‑based solution for face super‑resolution that uses higher resolution reference images to aid in the task. We show an alignment module based on the spatial transformer that is considerably more stable than the popular deformable convolutions. We also show an aggregation function that can take good quality information from the reference images when available or suppress the function when such information is unavailable. Finally, we show that our relatively smaller model can achieve state of the art results on multiple datasets. The source code is available at https://github.com/varun‑jois/FSRST.
Authors:Chuyifei Zhang
Abstract:
The test suites used as RLVR rewards for code have natural false positives: per‑task, persistent, asymmetric errors that accept the same wrong programs every time they appear, unlike the symmetric or resampled noise assumed by existing noise‑robustness analyses. We run a preregistered two‑arm causal contrast on a deployed suite: GRPO on identical MBPP tasks, seeds, and compute, rewarded by the original MBPP tests (leaky) versus the MBPP+ extra tests (hardened). Two further families replicate the design under a preregistration frozen before their data existed. [C] The average held‑out effect is bounded: non‑inferior under a preregistered 1.5‑pt margin (gap 0.20 pt, one‑sided 95% upper bound 0.75 pt). [C] Rewarded false‑positive mass tracks a cheap static leakiness audit computed before training (Spearman 0.80), and the registered train‑side test puts the leak‑stratum FP share +43.8 pt above clean tasks. [E] Auditing every rewarded FP under signed, human‑adjudicated rules finds a large residual of verified genuinely wrong code: 47.57% record‑weighted; both replication families reproduce a large share. The reward paid for real bugs, not merely suite artifacts. [E] Mechanism evidence is consistent with selection of pre‑existing error modes rather than learned exploitation: FP incidence does not grow within our horizon, and untrained base models already produce the same wrong outputs under the leaky filter. We then turn the same instrument on the frontier judges themselves: on their own false positives they self‑assess only weakly, a same‑author test is unresolved, and even the highest‑scoring reader we probe stays far below its score on a weaker policy's errors ‑‑ two subjects on MBPP, licensing nothing about frontier models in general. A cheap static audit locates exposure before training; hardening the reward removes the measurement inflation, though here it buys little capability.
Authors:Spencer Topel
Abstract:
Data re‑uploading parameterized quantum circuits (DRU‑PQCs) are universal function approximators, yet their expressivity produces oscillatory, non‑convex loss landscapes that resist gradient‑based optimization. We show that the primary optimization bottleneck in DRU‑PQCs is not insufficient capacity but a structural failure mode we term Fourier locking (FL): because encoding weights and entangling layers are nonlinearly coupled, random initialization on high‑frequency targets collapses the encoding parameters into spurious local minima. Two Fisher diagnostics characterize FL. The input‑space quantum Fisher information F_x measures the effective frequency content of the encoded state; the Fisher discriminant ratio of the measured features measures their alignment with the class labels. In two independent 50‑seed experiments, the locking is literal: trapped circuits hold F_x frozen for the entire run, while escaping circuits migrate their frequency content (direct training: r_pb = ‑0.48; curriculum: d = 1.34; both p < 0.001). The replicated signature is this spectral mobility, not any endpoint value of F_x, and trapped circuits retain a fully non‑degenerate parameter‑space QFIM (r_pb \approx 0): the failure is spectral misalignment of a responsive state, not a loss of geometric sensitivity. A frequency‑staged homotopy protocol that paces the target frequency (f: 1.0 \to 3.0) convexifies the early loss landscape; escaping circuits raise F_x in step with the curriculum, and the escape rate triples (18% vs. 6%). Fourier locking is a frequency‑alignment problem, and its remedy is frequency pacing.
Authors:Mingjie Xie, Guangjun He, Dongli Xu, Youtian Lin, Hongjue Li, Pengming Feng, Jian Guan, Yue Deng
Abstract:
Open‑vocabulary dense perception (OVDP) aims to localize objects unseen during training by leveraging textual knowledge. Despite the remarkable progress of recent CLIP‑based approaches, we identify a critical limitation: synonym‑induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns. This inconsistency undermines the robustness and performance of existing methods in real‑world OVDP applications. To address this issue, we propose SynCLIP, a Synonym‑Coherent Language‑Image Pretraining framework that enhances synonym‑robust grounding for OVDP. SynCLIP introduces a Semantic‑consistent Spatial Attention alignment (SSA) module to enhance spatial attention consistency by minimizing discrepancies between attention maps of original and synonymous expressions. Furthermore, a Spatial Attention Refinement (SAR) module selectively strengthens the most semantically relevant spatial regions within aligned maps for more precise and stable grounding. To support synonym‑coherent pretraining, we also construct a Synonym‑Enriched Visual Corpus (SEViC), which augments each category with multiple synonyms and textual definitions. Extensive experiments on multiple benchmarks demonstrate that SynCLIP substantially improves grounding consistency under diverse linguistic variants and achieves state‑of‑the‑art performance among CLIP‑based OVDP methods. Code is available at https://github.com/Justlovesmile/SynCLIP.
Authors:Jingxiang Zhang, Lujia Zhong, Zijie Zhu, Shuo Huang, Yuang Xu
Abstract:
Few‑shot multimodal classification commonly attaches a lightweight head, such as k‑nearest neighbors, logistic regression, or a linear SVM, to a frozen pretrained encoder. Although computationally efficient, these heads can produce poorly calibrated confidence scores, limiting their reliability in calibration‑sensitive applications. We evaluate TabPFN as a plug‑and‑play, zero‑gradient classification head for frozen image, text, and audio encoders. Across 22,820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log‑likelihood (NLL) and expected calibration error (ECE). At a representative setting, it reduces NLL by 48‑‑62% and ECE by 2.1‑‑5.3× relative to the average of the eight baselines while matching or exceeding their average accuracy. Its accuracy advantage is conditional, concentrating at moderate‑to‑high shot counts and low‑to‑moderate feature dimensions (k \ge 50, d \le 32), and diminishing when labeled data are scarce, feature dimensions are high, or competing methods approach ceiling accuracy. In targeted backbone‑adaptation experiments, replacing the trained linear head with TabPFN substantially improves calibration while preserving competitive accuracy. These results provide empirical guidance for using TabPFN as a training‑free head in calibration‑sensitive multimodal classification. To support transparency and reproducibility, we publicly release the source code, experiment configurations, and evaluation scripts in our GitHub repository: https://github.com/Jingxiang‑Zhang/tabpfn‑multimodal‑embeddings.
Authors:Yingji Zhong, Dave Zhenyu Chen, Fuzhao Ou, Youyu Chen, Zhihao Li, Lanqing Hong, Dan Xu
Abstract:
Recent generalizable 3D Gaussian Splatting models have advanced long‑sequence novel view synthesis (NVS), but at the cost of substantial redundant computation. We identify that the redundancy can be mitigated based on two observations: (i) high‑precision geometry is not strictly required for high‑quality NVS; (ii) appearance learning is generally easier than geometry recovery. Motivated by these insights, we propose an asymmetric architecture that decouples geometry and appearance modeling. The geometry branch processes coarse‑grained tokens with most of the parameters for multi‑view reconstruction, while the appearance branch operates on fine‑grained tokens to capture details using significantly fewer parameters. The two branches interact through bilateral connections, enabling mutual guidance for their respective tasks. This task‑aware asymmetry reduces the computational redundancy and allocates the computation more judiciously, thereby increasing parameter efficiency and enabling smaller models to achieve strong performance. On 32‑view 960P inputs, our model matches optimization‑based methods while delivering nearly 800x speedup, and surpasses the zero‑shot performance of state‑of‑the‑art generalizable models with markedly fewer parameters and reduced training/inference overhead, achieving an overall efficiency improvement.
Authors:Thanh-Nhan Vo, Thanh-Khoi Nguyen, Trong-Thuan Nguyen, Trung-Hoang Le, Minh-Triet Tran
Abstract:
Automated understanding of complex soccer scenarios from video remains a significant challenge for contemporary vision‑language models (VLMs), which suffer from shallow cross‑modal alignment and exhibit fundamental limitations in multi‑step reasoning and coordinated tool integration. We present TreeSoc, a structured reasoning framework that reformulates soccer video question answering as a hierarchical search problem rather than a single‑pass prediction. Specifically, TreeSoc employs a dynamic depth‑first search (DFS) mechanism that decomposes complex queries into sequentially ordered sub‑tasks, enabling iterative reasoning refinement through explicit intermediate states. This tree‑structured decomposition naturally supports adaptive tool routing, wherein domain‑specific modules are selectively activated and their outputs incorporated at each reasoning node to produce contextually grounded predictions. On SoccerBench, TreeSoc achieves state‑of‑the‑art performance, with accuracies of 85.2%, 87.4%, and 82.2% on TextQA, ImageQA, and VideoQA, respectively. Additionally, TreeSoc further demonstrates strong cross‑domain generalization, attaining 74.16% accuracy on NExT‑QA. These results establish structured, tool‑augmented tree reasoning as an effective paradigm for robust video understanding. Code is available at: https://github.com/thanhnhan29/TreeSoc.
Authors:Ananya Acharya, Trenton Goyette, Masoud Ataei, Adrian Stoica, Vikas Dhiman, Mohammad Javad Khojasteh
Abstract:
This paper introduces a hierarchical control architecture for multi‑agent adversarial environments, decoupling strategic task planning from rigorous safety assurance. The system formulates pursuit‑evasion as a zero‑sum receding‑horizon game, solved via an iterative minimax \aclmpc scheme. This allows pursuers to anticipate and block evader trajectories using transverse velocity penalties rather than relying on reactive heuristic formations. To guarantee collision‑free operation without compromising the convexity of the \aclmpc, a discrete‑time \aclcbf operates as an inner‑loop safety filter. Through simulated experiments, we demonstrate the framework's adaptability. By simply altering the weights of the shared zero‑sum payoff and \aclcbf constraints, the swarm can fluidly switch from aggressive pursuit‑evasion tactics to strict perimeter defense and area denial, demonstrating robust performance across varying rules of engagement without structural changes to the control logic. The source code is available: https://github.com/ananya‑ac/pursuit‑evasion‑mpc‑cbf.
Authors:Zheng Zeng, Deepak Sridhar, Nuno Vasconcelos
Abstract:
Vision‑language models (VLMs) such as CLIP enable zero‑shot classification by comparing image features with text prompts in a shared embedding space. A fundamental property underlying this capability is the global comparability of logits across arbitrary candidate classes. However, VLMs are often adapted to fine‑grained domains using techniques such as LoRA. While this improves in‑domain accuracy, out‑of‑domain accuracy degrades. This leads to a highly fragmented model ecosystem, with thousands of specialized models. Multi‑Expert‑Domain classification seeks to address this problem, by merging LoRAs trained independently on specialized domains. However, due to the independent training, the various domain experts no longer produce globally calibrated logits. As a result, when evaluating over the union of multiple domain‑specific class sets, heterogeneous logit scales induce cross‑domain interference and artificially high confidence for out‑of‑domain classes, inducing prediction errors. In this work, we identify domain supervision and cross‑domain logit miscalibration as the key issue to scalable multi‑domain zero‑shot recognition. We propose MED‑DSLC, combining domain supervised training and domain‑wise logit scaling, to explicitly restore global logit comparability. MED‑DSLC is a lightweight solution for MED classification, which is shown to preserve within‑domain discrimination while reducing cross‑domain logit interference with minimal data. Extensive experiments across diverse fine‑grained benchmarks demonstrate that it substantially improves mean accuracy (+15%), cross‑domain robustness, and scalability in the size of MED classification problem. Our results show that restoring output‑level calibration is essential under highly data imbalanced settings for achieving a truly zero‑shot VLM under multi‑domain specialization.
Authors:Cecilia Curreli, Florian Hofherr, Dominik Muhle, Abhishek Saroha, Riccardo Marin, Daniel Cremers
Abstract:
Existing Stochastic 3D Human Motion Prediction models are fundamentally constrained by hard‑coding the skeleton kinematics, severely limiting generalization, preventing cross‑dataset training, and requiring complex data retargeting. We introduce EquiFusion, the first kinematics‑agnostic model to solve this bottleneck, implementing a latent diffusion model with a permutation equivariant architecture. EquiFusion treats the kinematics' connectivity as an explicit input parameter, ensuring its internal computations are inherently agnostic to joint ordering and graph structure. This novel design enables truly cross‑dataset generalization to unseen kinematics and unlocks novel zero‑shot directions, such as motion prediction from partial or occluded observations and targeted limb generation. EquiFusion achieves state‑of‑the‑art results on major benchmarks, being up to 75% more compact than previous kinematics‑specific methods, while achieving faster training and inference. EquiFusion thus establishes a new, flexible standard for robust human motion prediction. Model and training code are available at https://ceveloper.github.io/publications/equifusion/.
Authors:Johannes Kruse, Kasper Lindskow, Michael Riis Andersen, Ryotaro Shimizu, Julian McAuley, Pierre-Alexandre Mattei, Jes Frellsen
Abstract:
We introduce NAILS (Normative Alignment of Recommender Systems via Internal Label Shift), a simple and scalable method for aligning recommendation outputs with target distributions over item‑level attributes, such as categories. Recommender systems optimized solely for user engagement often fail to satisfy broader normative objectives, including fairness, diversity, and editorial values. NAILS modifies the user‑conditional item distribution to induce a specified marginal distribution over attributes while preserving the preferences learned by an existing recommender system and requiring no model retraining. We formulate this problem as a form of label shift applied internally within a hierarchical classification framework. By adopting a stakeholder‑centric perspective, NAILS enables recommendation outputs to be aligned with global normative objectives. Empirically, we show that NAILS consistently improves attribute‑level alignment with minimal impact on user engagement, providing a practical mechanism for value‑driven recommendation.
Authors:Aqi Dong
Abstract:
3D Gaussian Splatting represents scenes as finite mixtures of anisotropic Gaussians whose number of components K is set by heuristic density control or user caps. Variational Bayes Gaussian Splatting (VBGS) recast splat fitting as conjugate variational inference, but K remains fixed. We replace the finite symmetric Dirichlet over mixture weights with a truncated stick‑breaking Dirichlet‑process prior ‑‑ and, as a theory‑backed alternative, a sparse overfitted finite Dirichlet ‑‑ so that the number of occupied components adapts to the data while every update remains a closed‑form coordinate‑ascent step; a natural‑gradient stochastic variant makes the per‑step cost independent of the number of points. We give an exact monotonicity guarantee, a rigorous truncation‑error bound correcting an anti‑conservative large‑α approximation in common use, and an honest account of what the fitted number of components estimates. Empirically: (i) the effective complexity \hatK adapts to scene complexity and recovers the true K within \pm 1 on well‑separated synthetic data with regime‑appropriate concentration; (ii) a deconfounded comparison shows the DP prior's contribution is complexity selection, not per‑component efficiency ‑‑ converged DP fits exceed single‑pass fixed‑K VBGS by +2.7 dB at matched budgets yet tie an equally converged fixed‑K baseline, and on 3D scenes DP‑Splat matches or exceeds VBGS's held‑out color prediction with 5.9‑7.6x fewer components; (iii) the posterior‑predictive color variance is well calibrated on model‑matched synthetic data; and (iv) the ordering suggested by exact‑posterior asymptotics reverses under mean‑field coordinate ascent: the DP prior resists over‑splitting while the sparse finite mixture saturates its truncation, a gap between variational practice and posterior asymptotics documented across three orders of magnitude in N.
Authors:Johannes Kruse, Ryotaro Shimizu, Kasper Lindskow, Jon Tofteskov, Michael Riis Andersen, Julian McAuley, Jes Frellsen
Abstract:
We present ZoRRO (Zero‑Weight Personalized Recommender System), a zero‑weight, training‑free framework for personalized news recommendation designed for scalable real‑world deployment. ZoRRO outperforms strong neural baselines in offline ranking evaluations and achieves click‑through rate performance in online A/B testing that is nearly on par with a state‑of‑the‑art deep learning model, while operating more than 600 times faster. Our experiments reveal gaps between offline and online performance and demonstrate that models with similar click‑through rate outcomes can produce markedly different recommendation distributions, thereby influencing the overall news flow. These findings position ZoRRO as a practical and efficient solution for large‑scale news recommendation and highlight the importance of evaluating recommender systems using metrics beyond accuracy alone.
Authors:Junchen Fu, Kaiwen Zheng, Ioannis Arapakis, Wenhao Deng, Xin Xin, Joemon M. Jose, Xuri Ge
Abstract:
Recently, large pretrained multimodal embedding models such as Qwen3‑VL Embedding have shown strong promise for sequential recommendation, as they provide reusable semantic item representations across modalities and domains. However, directly using these embeddings often leads to suboptimal performance because of domain misalignment. Efficient side adaptation is therefore an attractive solution. Although adapting all backbone layers should help, existing side adapters often degrade with depth, prompting layer dropping despite the loss of useful hidden states. This is due to two major challenges: (1) the lack of modeling in selecting fused representations during residual addition, and (2) the insufficient preservation of earlier representations during progressive sigmoid fusion. This paper therefore asks a practical question: How can we design a side adaptation approach that effectively unlocks the potential of large pre‑trained multimodal embedding models? To address this question, we propose Stresa, a stream‑aware side‑adaptation framework for frozen large pre‑trained multimodal embedding models in sequential recommendation. Stresa introduces Stream‑aware Hidden‑Adapter Fusion (SHAF) to preserve historical side memory during fusion and Residual Stream Adapter (ReSA) to produce selective residual updates across layers. Empirically, Stresa consistently outperforms standard side adapters and state‑of‑the‑art baselines on public datasets across multiple backbone embedding models. These results highlight the promise of adapting large embedding models for sequential recommendation. Our code is publicly available at https://github.com/GAIR‑Lab/Stresa.
Authors:Pei-Sze Tan, Sailaja Rajanala, Yee-Fan Tan, Raphael C. -W. Phan, Huey-Fang Ong
Abstract:
Micro‑expression recognition is limited by the small scale, narrow demographic coverage, and restricted emotion labels of existing datasets. We introduce EquiME, a synthetic micro‑expression dataset built from AU‑guided image‑to‑video generation. EquiME contains 75K videos generated from 15K source face images across five target emotions, together with automatically inferred demographic metadata and video‑quality measurements. We evaluate EquiME using frame‑pair similarity, spatial variation, and no‑reference perceptual‑quality metrics, together with cross‑dataset MER experiments on SAMM and CASME II. Models trained on EquiME achieve competitive cross‑dataset performance on SAMM and CASME II and show comparatively low variation across the four evaluated architectures. This paper focuses on the dataset design, the structured AU‑conditioning pipeline used for video generation, and the empirical evidence needed to assess EquiME as a synthetic MER resource. Project page: https://kirito‑blade.github.io/me‑vlm/
Authors:Peizhuo Li, Emre Aksan, Alexandru-Eugen Ichim, Thabo Beeler, Olga Sorkine-Hornung
Abstract:
Diffusion models faithfully reproduce their training distribution, but also inherit its imbalances and leave rare or under‑represented modes hard to reach. A natural inference‑time remedy is to sample from the high‑temperature target p^(γ)_0(x) \propto p_0(x)^γ for 0 < γ< 1, which flattens dominant modes and lifts rare ones. However, naive score scaling while correctly reweighting modes also inflates the per‑mode variance, breaking the reverse diffusion process and degrading sample quality. We introduce variance‑corrective time shifting, a training‑free fix that queries the network at a shifted timestep and scales the resulting score by γ, canceling the variance inflation while preserving the mode reweighting. The correction turns simple temperature sampling into a practical diversity knob for pretrained diffusion and flow‑matching backbones with no retraining, and we demonstrate consistent gains at minimal cost to sample quality and condition fidelity across DiT, Stable Diffusion and Motion Diffusion models. We further show that the timing of the temperature intervention enables coarse‑to‑fine control: high‑noise stages drive compositional diversity across modes, while low‑noise stages drive local appearance variation under a fixed composition.
Authors:Erdi Sayar, Ersin Daş, Joel W. Burdick, Alois Knoll, Erdal Kayacan
Abstract:
A key limitation on the use of diffusion models in robotic planning is their inability to inherently enforce safety or dynamical constraints, which often results in physically infeasible or unsafe outputs. Hybrid approaches that employ model predictive control (MPC) to address this problem can be unstable, as poor trajectory initializations from the diffusion model prevent the MPC from converging to a safe and feasible solution. To overcome these challenges, we propose D‑SafeMPC, which enhances the interaction between diffusion and control. Our method guides the reverse diffusion process with control barrier functions (CBFs) and control Lyapunov functions (CLFs) and employs an iterative‑projection scheme where an MPC refines the trajectory at each denoising step. This steers sampling toward safe, goal‑directed regions and provides reliable MPC warm starts. In simulations on a Franka manipulator across four scenarios (one static‑obstacle and three dynamic‑obstacle settings) and in a sim‑to‑real experiment on a physical Franka robot, D‑SafeMPC improves safety, task success rates, and planning efficiency over state‑of‑the‑art baselines. To facilitate reproducibility, our source code and experimental configurations are available in a repository at https://github.com/erdiphd/D‑SafeMPC
Authors:Venkanna Babu Guthula, Oswin Krause, Dimitri Gominski, Hui Zhang, Johan Mottelson, Ankit Kariryaa, Nico Lang, Christian Igel
Abstract:
Supervised learning for image segmentation typically requires spatially aligned image and label sets. When images and labels originate from different sources, the pairing may be misaligned, which can significantly deteriorate the performance of the learned models. This is especially common in remote sensing, when aerial or satellite images are co‑registered with labels from another source (e.g., OpenStreetMap). In this work, we propose a novel approach for training on misaligned labels, where we simultaneously learn the label alignment. Our align and segment (AnS) approach builds on the spatial transformer module to transform the misaligned labels using an affine transformation to provide a better learning target for a canonical semantic segmentation network. We prevent shortcut learning of misaligned labels in these semantic segmentation networks through a self‑supervised regularization loss and show that it is complementary to data augmentation, especially for systematically misaligned training data. A decisive characteristic of our AnS approach is that it learns without requiring any golden labels. We experimentally show on both synthetic and real‑world data from different cities that our approach enables high‑quality building segmentation and precise label‑image alignment at the same time. Code and derived datasets are available at https://github.com/venkanna37/align‑and‑segment
Authors:Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad Vittala
Abstract:
Quantifying abstractiveness in generated summaries is essential for evaluating summarization models beyond surface‑level metrics like ROUGE. We introduce Reference Abstraction (RA), Summary Abstraction (SA), and Abstraction Ratio (AR) ‑‑ a set of principled heuristic metrics that measure how much a summary diverges from extractive copying of the source text. The formulation uses the harmonic mean of document lengths modulated by a cubic non‑overlap factor, yielding dimensionally consistent, bounded output with non‑linear sensitivity to the extractive‑abstractive boundary. Evaluation on 100 XSUM documents across four summarization models (BART‑large‑cnn, Pegasus‑xsum, DistilBart, MT5‑small) demonstrates that the metrics successfully discriminate between extractive models (SA ~ 0.12‑0.26) and abstractive models (SA ~ 0.96‑1.77), and that the Abstraction Ratio identifies summaries requiring manual evaluation for potential hallucination. Code and results are available at https://github.com/katweNLP/AbstractionStudy.
Authors:Saadeldine Eletter, Owais Aijaz, Preslav Nakov
Abstract:
Multimodal retrieval‑augmented generation (RAG) is often evaluated with clean evidence, yet real retrieval can return topically relevant but unreliable content: false text and misleading images from corrupted metadata, entity swaps, typographic overlays, semantic edits, adversarial patches, blends, or style transfer. We introduce QIMG‑7, a controlled benchmark for multimodal retrieval pollution in multi‑sentence factual QA, spanning four datasets, seven image‑attack families, and 16 paired clean/polluted regimes, for 1,760 evaluation rows per method. Across four generator/gate stacks, naive multimodal fusion is brittle: in the main gpt‑4o‑mini stack, Full‑MM support drops from 0.908 with clean text to 0.490 with polluted text, often making Parametric fallback safer than retrieval. We propose source‑aware trust resolution (SATR), a training‑free approach that compares Parametric, Text‑only, and Full‑MM candidate answers and selects among candidate answers or falls back based on source reliability. The Field‑Selector variant achieves the best balanced score, 0.816, improving over Full‑MM by 11.7 points and over the Cascaded Router by 2.7 points. Ablations show that, in this text‑first setting, explicit text‑reliability modeling is the dominant driver of these gains. Overall, in text‑first factual QA with multimodal retrieval conflict, our results support selective trust rather than unconditional fusion. Artifacts are available at https://github.com/SaadElDine/Trust_Before_Fusion.
Authors:Yi Zhao, Jiajun Gao, Chenyang Xu, Yuxi Zhou, Hao Wang
Abstract:
Deploying deep learning models for automated electrocardiogram classification on resource‑constrained wearable devices remains challenging due to high computational costs. To address this, we propose LSTrans, a lightweight hybrid model designed for efficient and sensitive ECG analysis. LSTrans introduces a specialized 1D convolutional backbone with an interleaved layer architecture to capture both macroscopic rhythmic trends and microscopic morphological variations. This backbone is cascaded with a Transformer encoder to model long‑range temporal dependencies, incorporating Low‑Rank Adaptation across critical layers to compress the model and reduce the trainable parameter space. We further employ homogeneous and heterogeneous knowledge distillation to transfer diagnostic expertise from high‑capacity teacher models to the student. Experimental results on multiple benchmark datasets demonstrate that LSTrans achieves a competitive balance between diagnostic sensitivity and resource efficiency, substantially reducing peak memory footprints and training latency during downstream adaptation. The source code is available for review at https://github.com/zyee00128/LSTrans4BIBM.
Authors:Dung Minh Do, Nhat-Thanh Huynh, Duc Minh Huynh, Doanh C. Bui, Khang Nguyen
Abstract:
Whole‑slide images (WSIs) provide rich tissue‑level and cellular‑level information, but storing and transmitting high‑magnification pathology data is resource‑intensive. Moreover, annotating WSIs at the pixel level is labor‑intensive and time‑consuming. Therefore, it is important to investigate whether low‑magnification pathology images with limited annotations (i.e., image‑level instead of pixel‑level labels) can achieve performance comparable to high‑magnification images. This paper presents a systematic benchmark study on weakly supervised histopathological image segmentation under different low‑resolution storage settings. Starting from high‑resolution image patches, we simulate lower‑magnification inputs and reconstruct them to the original size using interpolation and deep learning‑based reconstruction methods before applying the weakly‑supervised segmentation pipeline. This framework enables a quantitative evaluation of how weakly supervised methods respond to different levels of resolution degradation. Experimental results show that reconstruction quality metrics alone are insufficient to predict downstream segmentation performance. In particular, the study identifies a critical degradation point where the localization of small‑scale structures declines significantly. These findings provide practical guidance for designing efficient digital pathology storage systems while maintaining reliable automated analysis. Code is available at https://github.com/Dung‑Dx/LowMagWSS
Authors:Siyuan Song, Zhiheng Qian, Yunhao Zhang, Linyang He, Xiaozhe Ji, Yingxin Lin, Hongao Zhu, Chongtian Shao, Chuhan Lang, Luan Li, Rui Wang, Renfen Hu, Shaonan Wang, Hai Hu
Abstract:
This paper describes the first ChineseBabyLM challenge, which will be held in the 2026 NLPCC conference. The challenge calls for researchers to train language models from scratch with 100 million Chinese tokens and evaluates the models on 3 tracks of tasks: NLU, cognitive alignment and Hanzi knowledge. There is no restriction on tokenizer, model architecture and the number of training epochs. Details of the challenge can be found in https://chinese‑babylm.github.io/.
Authors:Fengji Zhang, Tianyu Fan, Yuxiang Zheng, Xinyao Niu, Chengen Huang, Jacky Keung, Bei Chen
Abstract:
Recent advances in equipping Large Language Models (LLMs) with search tools and outcome‑reward reinforcement learning (RL) have achieved new state‑of‑the‑art results on open‑domain QA tasks. However, we argue that current training paradigms harbor a critical vulnerability: they predominantly reward correct answers but fail to penalize fabricated ones when retrieval fails, thereby implicitly exacerbating hallucinations. To address this, we propose Abstention‑Aware Reinforcement Learning (AWA‑RL), which dynamically shapes the abstention reward utilizing the model's query‑specific prior capabilities and continuous on‑policy training observations. We also introduce a novel metric, RA‑F1, to measure the capability‑reliability trade‑off. Compared to non‑abstaining baselines, AWA‑RL boosts absolute precision by up to 10.3% and overall RA‑F1 by 2.9%, with only marginal sacrifice in raw accuracy. These results confirm that AWA‑RL successfully yields highly capable and reliable search agents. The code, data, and model weights are publicly available at https://github.com/zfj1998/AWA‑RL.
Authors:Mohannad Takrouri, Nicolas M. Cuadrado A., Martin Takáč
Abstract:
The accelerating shift toward low‑carbon power systems, together with the widespread adoption of behind‑the‑meter technologies such as rooftop solar and electric vehicles, is placing new operational and analytical demands on electricity grids. At the same time, smart‑grid research increasingly relies on machine learning (ML), yet progress is constrained by limited access to high‑resolution household energy data due to privacy concerns, regulatory barriers, and collection costs. This work presents WattCouncil, a data‑generation framework in which household electricity demand is generated by a council of Large Language Model (LLM)‑based agents operating in specialized roles to generate, audit, and validate structured energy scenarios under explicit cultural, temporal, and physical constraints. Rather than acting as static predictors, these agents serve as adaptive decision‑makers within a governed pipeline. Motivated by studies highlighting the importance of contextual factors in energy use, our framework produces context‑sensitive daily routines through a guided reasoning process that incorporates household composition, temporal factors, and environmental conditions. We evaluate the generated profiles against the detailed CER dataset, which contains over a year of load measurements for 4232 households together with survey‑based socio‑economic information. We further assess the consistency of the framework through ablation studies. Source code is available at https://github.com/Singularity‑AI‑Lab/wattcouncil
Authors:Haojie Huang, Zhang Ye, Linfeng Zhao, Boce Hu, Mingxi Jia, Yu Qi, Ahmed Agha, Dian Wang, Robert Platt, Robin Walters
Abstract:
The action space poses a major challenge in robot learning, since it is often high‑dimensional, can span long time horizons, and frequently admits multi‑modal optimal solutions. A good choice of action representation and loss function can help to address these concerns, but there are often trade offs. We propose Action Map Policy (AMP), which casts 3D closed‑loop manipulation policy learning as a classification problem in image space. While classification has been an effective formulation in generative language models, applying it to robot action learning is difficult because naively discretizing high‑dimensional continuous actions explodes the token vocabulary. Our key idea is to project 3D actions onto the camera image planes and treat each pixel location as a discrete class, thus controlling dimensionality while retaining multi‑modality. This method supports millimeter‑level precision for high‑dimensional actions without requiring a prohibitively large vocabulary, while preserving fine‑grained pixel‑wise visual signals. Furthermore, it can predict the entire action chunk in a single forward pass, avoiding complex noise scheduling and iterative denoising while achieving substantially faster inference than diffusion policies. Experiments on various manipulation tasks show that AMP outperforms strong baselines, achieving higher success rates, faster inference, and enhanced spatial reasoning.
Authors:Caihui Yan, Gang Cao, Huawei Tian, Zhen Li, Yuhang Zhai
Abstract:
The rapid advancement of generative artificial intelligence (AI) has made synthetic images remarkably realistic, posing security threats such as misinformation and fraud. It is significant to detect the synthetic image in the manner of passive and blind image authentication. Most existing detectors rely on supervised training with large labeled datasets, leading to high costs and degraded performance on unknown generative models. To attenuate such deficiencies, we propose a training‑free detection method. Specifically, noise residual fingerprints are first extracted by a simple yet effective pre‑trained Noiseprint++ model. Then multi‑scale features are further extracted from such residual by a frozen Vision Transformer (ViT), followed by adaptive weighted fusion. Only a few real image samples are used needed to initialize the clustering centers for unsupervised K‑Means, distinguishing real and synthetic images without training. Extensive evaluations on four benchmark datasets show that our proposed scheme achieves an average accuracy of 82.2%, outperforming the state‑of‑the‑art detectors on generalization ability. Superior performance is gained on the popular diffusion type of synthetic images, and the effectiveness of each module is validated by ablation studies. Source code will be publicly available at https://github.com/multimediaFor/NoiseCluSID.
Authors:Matthias M. M. Buehlmaier
Abstract:
In pre‑LayerNorm looped transformers, LayerNorm inside the recurrent block acts as an implicit gain controller: by coupling the block's local Lipschitz constant inversely to the activation scale, it renders the recurrence Jacobian non‑normal ‑‑ asymptotically contractive at every verified fixed point even where its operator norm exceeds 1 ‑‑ so the true stability budget is the spectral margin, not an operator‑norm bound. That margin depletes as the carry ρ\to 1, and a minority of initializations never converge to a fixed point at all, so the diagonal carry constraint ρ(\barA) < 1 is necessary but not sufficient for convergence of the full recurrence. Training experiments across six tasks, including a controlled ablation, reveal that the linear carry is not the depth‑memory mechanism: gradient descent routes memory through the block's more expressive nonlinear recurrence and leaves the stability‑constrained carry at rest ‑‑ the carry's role is stabilization, not memory. We characterize the boundary of this claim: on tasks with axis‑aligned per‑channel structure, gradient descent does recruit the carry. All results are derived analytically and verified in a from‑scratch, CPU‑scale implementation; verification at larger scale is needed.
Authors:Zipeng Gao, Zhi Zheng, Qingrong Xia, Junda Lin, Ziwei Zhao, Tong Xu, Zhefeng Wang, Enhong Chen
Abstract:
Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory‑bound bottlenecks. However, traditional speculative decoding typically relies on auxiliary draft modules, incurring significant training and communication overhead. Although recent methods attempt to generate drafts within the target model itself, they often fail to fully exploit its latent parallel capacity due to a lack of structural coordination. In this paper, we propose Progressive Tree Drafting (PTD), which employs a structured, guided parallel drafting strategy to harness the model's parallel potential. By coupling a progressive tree structure with a stepwise pruning mechanism, PTD actively guides the LLM to explore multiple semantic paths in a single forward pass, ensuring both draft diversity and coherence. Experiments demonstrate that PTD achieves up to 2× decoding speedup across various benchmarks while remaining training‑free and model‑agnostic. Our code is available at: https://github.com/MINE‑USTC/PTD.
Authors:Ilia Karpov
Abstract:
An LLM agent's public behaviour reveals little about its social reasoning: an agent that votes correctly may be guessing, and an agent that lies well leaves no trace of what it actually believes. We present MafiaScope, an open testbed that turns the social deduction game Mafia into a measurement instrument for machine Theory of Mind. After every public utterance, every agent privately answers a configurable set of structured probe questions; the answers never re‑enter the game and are scored automatically against the ground truth the engine knows. An interactive visualizer renders the belief trajectories: impersonate mode shows the game as one agent sees it, panels chart timeline‑aligned accuracy and calibration, and counterfactual replay forks any recorded step. In a 32‑game DeepSeek case study with 13,815 parsed probe answers, stated confidence is poorly calibrated, with expected calibration error 0.17, agents over‑predict being suspected 1.5 times, and a 30‑fork replay experiment walks the counterfactual replay workflow end to end. Engine, viewer and a corpus of 200+ cross‑model games are released under an open licence; live demo: https://karpovilia.github.io/mafiascope/; screencast: https://vimeo.com/1208920221.
Authors:Zhaoyang Li, Yanjun Li, Wangkai Li, Yujia Chen, Tianzhu Zhang
Abstract:
Vision‑Language Models (VLMs) are costly at inference time because they must process long sequences of visual tokens. Existing token pruning methods often degrade under high compression by blindly discarding information, breaking spatial structure or collapsing diversity. We propose SpecFlow, a training‑free framework that shifts the paradigm from destructive pruning to conservative condensation, strictly enforcing spatial coverage and statistical conservation to ensure stability. Treating visual tokens as nodes in a kNN graph, SpecFlow (i) computes a stable importance field via spectral heat flow to preserve structural coherence, (ii) allocates budgets via adaptive spatial partitioning to guarantee coverage, and (iii) aggregates discarded information into coreset sinks to maintain statistical conservation. The method is plug‑and‑play, requires no fine‑tuning, and is compatible with FlashAttention. Experiments confirm that our SpecFlow outperforms SOTA methods across tasks, VLM architectures, and pruning ratios. Notably, LLaVA‑1.5 with SpecFlow retains 95.6% of original performance despite pruning 88.9% of visual tokens, offering an exceptional efficiency‑accuracy balance. Code is available at https://github.com/Lzy‑dot/SpecFlow
Authors:Jun Chen, Erdent Bao, Wenlong Dong, Jierui Liu, Qi Cai, Hao Wan, Shaopeng Li, Weijun Qin, Jing Liang, Huiping Zhuang
Abstract:
Language‑conditioned Imitation Learning (IL) is essential for enabling robots to perform complex tasks following natural language instructions. However, generalizing to multi‑step compositional tasks remains a significant challenge. While hierarchical approaches attempt to address this by decomposing tasks into atomic skills, existing methods often suffer from training instability and codebook collapse due to the tight coupling between high‑level skill reasoning and low‑level action generation in joint training paradigms. Inspired by the Dual‑Process Theory of cognition, we propose Dual‑Process Atomic Skill Learning (DASL), a novel asynchronous hierarchical imitation learning framework that decouples slow semantic reasoning from fast, real‑time motion control. DASL comprises a Slow‑Frequency Policy that predicts interpretable, discrete skills via Vector Quantization, and a High‑Frequency Policy that leverages a latent diffusion model and a Decision Transformer to generate precise actions conditioned on these latent skills. By asynchronously coordinating these modules and utilizing diffusion to structure the latent space, our framework mitigates the skill codebook interference problem common in joint training paradigms. Evaluations across simulation benchmarks and experiment demonstrate that DASL significantly outperforms state‑of‑the‑art baselines, excelling in skill acquisition and compositional generalization to unseen instructions. GitHub page: https://github.com/Hatakekaka/DASL
Authors:Muhammad Awais Bin Adil, Saad Aamir
Abstract:
The lineage graph of open‑weight language models is self‑reported: Hugging Face's base_model metadata field is optional and unverified, and over 60% of Hub models document no parentage at all. Methods for detecting lineage from weights exist in the research literature, but each ships as paper code tied to one signal and one experiment; when a provenance dispute breaks, the analysis is redone by hand. This report describes modelDNA, a tool that fingerprints a model from roughly 100‑300 MB of ranged HTTP reads (instead of a full 15 GB download for a 7B model), compares the fingerprint against a reference database of foundation models across four published signal families, and returns one of eight verdict classes with a calibrated probability, preferring honest abstention to confident error. On a benchmark of 15 real Hub models with org‑documented parentage, judged against 8 candidate bases (13 positives, 107 hard negatives), the system achieves AUROC 1.0, zero false positives at its reporting threshold, and 13/13 correct top‑1 parent attribution. The report's second contribution is merge decomposition. Every mainstream weight‑merging method is (near‑)linear per tensor, and fingerprint sample positions are deterministic functions of tensor identity, so a merged model's fingerprint is the same linear combination of its parents' fingerprints. Mixture weights can therefore be recovered from fingerprints alone by sum‑to‑one constrained least squares. Against merges with published mergekit configurations as ground truth, the method recovers a slerp merge's layer‑interpolation curves at r = 0.999 and a dare_ties merge's mixture weights to within 0.011 of the published values, without downloading any weights beyond the fingerprints. All fingerprints, benchmarks, and the inferred lineage graph of 55 models are public and reproducible offline.
Authors:Khush Kataruka, Harshit Maurya, Anuja Vats, Murari Mandal, Kiran Raja, Praveen Kumar Chandaliya
Abstract:
Efficient waste segregation is critical for sustainable urban management and environmental governance. Existing automated systems are limited by single‑modality visual processing, insufficient contextual understanding, and weak regulatory alignment. To address these issues, we propose a language‑guided vision‑AI framework that integrates vision‑language models and multimodal large language models for joint visual‑linguistic reasoning. This framework implements a visual question answering paradigm aligned with India's Solid Waste Management Rules 2016. We construct a new WasteVQA dataset with 13,500 question‑answer pairs across 21 waste categories. Experiments show that the BLIP‑based model achieves a BLEU score of 0.8291 and a BERTScore of 0.9273, outperforming traditional CNN‑based methods. This work improves source‑level segregation accuracy, ensures regulatory compliance, and supports scalable deployment for municipal and citizen‑facing waste management, promoting multimodal AI in sustainable urban infrastructure. The source code and dataset are available at: https://github.com/Khushkataruka/WasteAssistant
Authors:Yixiong Chen, Alan Yuille
Abstract:
Large Language Model (LLM) agents are commonly trained from expert trajectories using supervised fine‑tuning (SFT), which treats multi‑turn agent behavior as ordinary text imitation. This recipe is simple and low‑cost, but it only learns to imitate the sequence of expert actions, rather than training the agent to choose the right action against plausible mistakes at each state. Existing methods to mitigate this problem include preference learning or reinforcement learning, but they usually need high‑cost environment rollouts and reward models. We propose Agentic‑DPO, a lightweight offline agent policy optimization method that turns expert trajectories into state‑conditioned preference supervision. At each expert action state, Agentic‑DPO samples a one‑step action from the current state, treats plausible wrong actions as negatives, and contrasts them with the expert action using a DPO‑style preference objective. To avoid mixing both policy and schema in preference learning, we introduce Policy‑Preserving Augmentation (PPA), which renders the same latent trajectory under multiple schemas while keeping the expert policy fixed. Agentic‑DPO requires no online environment rollout, reward model, or full‑trajectory student exploration. We conduct experiments across StableToolBench, tau‑bench retail, and Mind2Web, where Agentic‑DPO consistently improves agents at different model scales beyond imitation. In particular, it raises tau‑bench accuracy from 21.7% (SFT) to 41.4% for a 9B model, matching online GRPO under the same backbone with only step‑level rollouts and without environment interaction during gradient steps. The results suggest that expert trajectories can support low‑cost agentic policy optimization when converted from demonstrations into state‑level action preferences. Code for Agentic‑DPO is released at https://github.com/Schuture/Agentic‑DPO.
Authors:Vu Minh Tran, Khang Nguyen
Abstract:
UAV‑based vehicle re‑identification (ReID) has emerged as a promising technique for traffic surveillance, urban monitoring, and public‑safety applications thanks to the flexible viewpoints and wide‑area coverage provided by unmanned aerial vehicles. However, despite recent progress on UAV‑based vehicle ReID benchmarks, the robustness of existing methods under adverse weather remains insufficiently studied. This is important because weather degradation can significantly affect the fine‑grained appearance cues required for reliable vehicle matching in aerial imagery, especially under small object scale, viewpoint variation, and complex backgrounds. In this paper, we present a controlled comparative study of three representative recent vehicle ReID methods, namely CLIP‑ReID, MSINet, and AdaSP, on two UAV‑based benchmarks, VRU and UAV‑VeID. To ensure consistent robustness evaluation, we generate synthetic foggy and rainy variants of both datasets using an analytical weather‑effect pipeline while preserving the original identities and data splits. All methods are then trained and evaluated under matched clean, foggy, and rainy conditions. Experimental results show that adverse weather consistently degrades retrieval performance across both datasets, with rain causing larger drops than fog in nearly all settings. Among the evaluated methods, AdaSP demonstrates the strongest robustness, achieving 93.0% and 88.5% mAP on VRU‑Large, and 88.7% and 76.2% mAP on UAV‑VeID‑Test under foggy and rainy conditions, respectively. Overall, our findings show that simulated adverse weather substantially increases the difficulty of UAV‑based vehicle ReID, reveals clear robustness differences among recent methods, and highlights the need for weather‑aware model design and evaluation protocols in future aerial ReID research. The code is released at https://github.com/tranminhvu945/Benchmarking‑ReID.
Authors:Chunwei Ma, Russell Wolfinger
Abstract:
Existing hypotheses represent a concept in an LLM as a single point, a linear direction, or a Gaussian cluster, yet it remains unclear how and why such structures emerge. Here, we show that concept geometry can be precisely characterized via Laguerre Geometry, in which a concept is defined as a region‑‑a Laguerre‑Voronoi cell or a union of cells‑‑allowing us to strictly define, measure, and separate concepts. Building on this formulation, we show that finer‑grained concept structures, such as inclusion and hierarchy, are naturally revealed by the Laguerre weights. We then push this geometry inside the transformer. Decomposing each layer into piecewise‑linear operators, we show that a token's hidden trajectory is governed by two coupled mechanisms: a static tree of self‑contained piecewise‑linear flow, and a dynamic transport that hops the trajectory across trees when cross‑token attention fires. This decomposition yields Geometric Lens, a training‑free, hyperparameter‑free method for reading out the exact concept a hidden vector encodes at any layer. We also develop Laguerre Autoencoder, a 2D visualizer that renders both the decision geometry and a model's full reasoning trajectory in one view. Finally, we move beyond explanatory geometry toward actionable interpretability, showing that Geometric Lens recovers the correct factual token when a model is prompted with in‑context interference. The code is available on GitHub.
Authors:Hong Yang, Qi Yu, Travis Desell
Abstract:
Modern coding agents expose multiple tool surfaces ‑‑ IDE primitives, bash, and Model Context Protocol (MCP) code‑execution ‑‑ and the field has shipped three contradictory claims about which one matters. We run the missing crossed comparison: an integrity‑clean three‑arm ablation (baseline / bash_only / code_only) on synthetic computation tasks and SWE‑bench Mini modification tasks, holding model, harness, and prompts fixed, with two agents (Claude Code, OpenAI Codex CLI) so the comparison spans both regime and agent‑design axes. Across the four resulting (regime, agent) cells, restricting the agent to a single execute_code MCP tool is cheaper than ‑‑ or statistically tied with ‑‑ its cheapest tool‑rich rival in three cells (significantly on Artifact/Claude and SWE‑bench/Codex; directionally on Artifact/Codex), with pass rates statistically tied within each cell. The lone exception is SWE‑bench/Claude, where code_only is directionally costlier (+14.4%, not significant); a conditional‑cost analysis localizes that gap to failure‑cost on doomed‑run trajectories, not a per‑edit tax on successful runs. Two implications: the cheapest tool surface is jointly determined by task regime and agent design rather than by either axis alone, and the headline cost signal lives in cache‑adjusted cost ‑‑ not pass rate, which is invariant across surfaces at the model sizes we evaluate. The benchmark harness, task suite, and analysis code are available at https://github.com/hyang0129/onlycodes.
Authors:Shijin Wang, Zichong Chen, Yang Zhou, Hui Huang
Abstract:
Mesh deformation, the process of altering the vertex positions of a 3D mesh while preserving its topological structure, is a cornerstone of computer graphics. Despite the recent emergence of numerous text‑guided 3D mesh deformation methods, deforming an initial mesh into one that both adheres to text prompts and preserves its pose remains challenging. This paper proposes PoseAlign, which decomposes text‑guided mesh deformation into two stages: global pose scaling and local detail sculpting. Specifically, in the first stage, we introduce the Laplacian as a differentiable mesh representation to enable more efficient yet smoother global deformation. Then, we propose a novel pose‑aligned SDS loss by adapting score distillation sampling (SDS) with an attention‑sharing mechanism, which sculptures fine‑grained geometric details for the deformed mesh while preserving its original pose. PoseAlign significantly enhances the controllability of the overall deformation process, achieving a favorable balance between pose preservation and text alignment. Experiments demonstrate the competitive advantages of our method in text alignment and mesh quality. Code is available at: https://cousingrade6.github.io/PoseAlign
Authors:Xiaolei Hou, Zheng Pan, Hua Lan, Zhenghao Zou, Yinhong Chen, Chenxi Zhu, Yang Lyu, Jinwen Hu, Chunhui Zhao
Abstract:
Efficient exploration and target search in large‑scale unknown environments remain challenging for aerial robots due to the demands of broad spatial coverage, fine‑grained perception, and real‑time decision‑making. This paper presents SLIDER, a lightweight and memory‑efficient framework that avoids reliance on globally dense maps by combining a local sliding map with sparse global history information. A novel observation quality evaluation method is proposed, leveraging historical poses and sensor models to assess point cloud data in real‑time, enabling efficient frontier detection. To support scalable and responsive planning, an incremental viewpoint clustering strategy dynamically adapts to local updates, significantly reducing the number of candidate targets and decreasing computational load. A sparse global topological map is incrementally maintained to assist global planning and cost evaluation. Extensive simulations and real‑world experiments demonstrate that the proposed system outperforms state‑of‑the‑art methods in memory usage, decision latency, and search efficiency.
Authors:Zhihao Yao, Yuxuan Gu, Jixuan Yin, Bo Li
Abstract:
Pseudo‑labeling based on Optimal Transport (OT) has become an effective mechanism for enhancing short text clustering. Existing OT methods are short in modeling semantic consistencies between samples, which may assign different pseudo‑labels to semantically similar samples. These erroneous pseudo‑labels can cause the model to produce inferior clusters. This paper proposes a novel short text clustering framework, which remedies the neglect of semantic consistency in existing OT methods, generating reliable pseudo‑labels to facilitate clustering. Specifically, the proposed approach first designs an instance‑level attention mechanism to capture semantic relationships between samples, which are then integrated into the OT formulation to endow the transport process with neighborhood semantic awareness. By solving the proposed OT formulation, reliable pseudo‑labels are obtained that simultaneously account for sample‑to‑sample semantic consistency and sample‑to‑cluster global structure information. These pseudo‑labels are then used as supervisory signals to guide the model to achieve accurate clustering. Extensive experiments demonstrate that the proposed approach outperforms state‑of‑the‑art methods. The code is available at: \hrefhttps://github.com/YZH0905/CAOT‑STChttps://github.com/YZH0905/CAOT‑STC.
Authors:Jian Ning, Qin Zou, Linchun Wu, Yuanhao Yue, Kunmo Li, Shoubin Chen, Zhongyuan Wang
Abstract:
3D point cloud anomaly detection plays a vital role in industrial manufacturing, yet it faces significant challenges due to the scarcity and high acquisition cost of real anomalous samples. The inherently anomaly‑free training data further hinders detection methods from effectively learning discriminative features between normal and abnormal instances. To address these issues, we propose PA3AD, a novel framework that introduces a physics‑inspired pseudo‑anomaly generation strategy to create physically plausible anomalous samples from normal data. Additionally, we incorporate prototype features via a weight‑sharing mechanism to guide the model in capturing the distribution shifts between normal and anomalous samples. Specifically, PA3AD introduces two key innovations to tackle the scarcity of real anomalies. First, a physics‑inspired module generates diverse pseudo‑anomalous point clouds from normal data via multi‑physics modeling. Second, momentum‑updated prototypes and a difference‑aware fusion block capture stable normal representations and their discrepancies with pseudo‑anomalies. This design effectively learns distribution shifts, achieving superior detection performance. Extensive experiments on the Anomaly‑ShapeNet and Real3D‑AD datasets demonstrate that our method consistently outperforms existing state‑of‑the‑art approaches. Our code will be made publicly available at https://github.com/NingxiaoJian/PA3AD.
Authors:Yan Lin, Yuyang Dai, Jiahui Geng, Yuxia Wang
Abstract:
Existing approaches to infer user traits and generate responses consistent with a persona rely on static prompting. They lack calibrated uncertainty, ignore sequential evidence, and drift during long interactions. We present AI YOU, a framework that continually updates a personality profile with 22 dimensions from conversation and embodies it in a personal digital twin. Practically, the system combines prompting, Bayesian updating, and conformal prediction for persona inference. A periodically refreshed memory anchor and cognitive memory with three layers preserve persona consistency over long interactions. Across the main results, AI YOU \emph(i) achieves conformal coverage ranging from 0.921 to 0.976, \emph(ii) improves uncertainty calibration and reasoning grounded in memory, and \emph(iii) enhances persona fidelity over static prompting in role playing over 100 turns while reducing trait drift, for most evaluated backbones under adversarial settings with multiple agents. The prototype \emphAI YOU Town initializes an imaginative twin world for future interaction. The online demo is available at \hrefhttps://quinnnnnne‑ai‑you.hf.space/\mbox\textttquinnnnnne‑ai‑you.hf.space.
Authors:Ryota Kimura, Sangheon Park, Natalia Polouliakh, Taketo Akama
Abstract:
Dance‑to‑music generation is a promising task for applications such as choreography support and automatic accompaniment, where temporal coordination between body movement and sound is essential. In particular, using human joint positions as the motion representation is attractive because they explicitly capture body dynamics while being lightweight, privacy‑preserving, and easy to integrate with motion capture and pose‑estimation pipelines. A central challenge in this setting, however, is the scarcity of high‑quality paired dance‑music data, since collecting accurately synchronized pairs is costly and often constrained by copyright and performance rights. This makes it difficult to train end‑to‑end models solely from paired data. To address this issue, we propose a dance‑conditioned music generation framework that efficiently exploits both unpaired and paired data. Our method combines pretrained unimodal encoders for motion and music, beat‑guided contrastive pretraining to align their feature spaces, and a ControlNet‑style conditioning module on top of a pretrained text‑to‑audio diffusion model. Experiments on AIST++ demonstrate that the proposed techniques improve both dance‑music alignment and audio quality, as confirmed by quantitative and qualitative evaluations. Compared to a state‑of‑the‑art method, our approach achieves superior dance alignment performance and competitive audio quality. Code is available at https://github.com/kmraven/AudioLDM‑ControlNet .
Authors:Yunpeng Hong, Chenyang Bu, Di Wu, Yi He, Xindong Wu
Abstract:
Multimodal Entity Alignment (MMEA) aims to identify equivalent entities across different modalities. While existing methods enhance MMEA performance through black‑box context engineering strategies, their reliance on LLM parameter capacity and lack of theoretical interpretability remain unresolved. To this end, we first theoretically validate the mathematical equivalence between context engineering and model fine‑tuning in MMEA tasks, demonstrating that prompt components simulate contrastive learning‑based sequential fine‑tuning in MMEA. Building on this foundation, we then propose PTFEA, a curriculum‑learning‑inspired framework that translates fine‑tuning strategies into interpretable context engineering. Specifically, adaptive difficulty modulation dynamically adjusts information injection stages using confidence thresholds, establishing mathematical equivalence between curriculum learning weights and context sample selection; and three‑stage progressive inference incorporates entity information from simple to complex cases, mirroring the gradient descent process in fine‑tuning. Experiments on five public datasets demonstrate that PTFEA consistently outperforms strong baselines. In particular, on the ICWIKI dataset, PTFEA narrows the H@1 gap between Qwen2.5‑72B and 14B to 0.6%. Moreover, compared with the representative context‑engineering‑based MMEA method MM‑ChatAlign, PTFEA reduces the runtime of Qwen2.5‑72B from 21 hours to 1 hour and lowers token consumption from 2200‑3000 to 200‑400, achieving over 80% reduction on the ICWIKI dataset. This work provides the first theoretical framework unifying context engineering and fine‑tuning in MMEA, paving the way for future research that seeks to translate additional fine‑tuning strategies into context engineering paradigms. Our code is available at https://github.com/DMiC‑Lab‑HFUT/PTFEA.
Authors:Seyed Arshan Dalili, Ajay Narayanan Sridhar, Vijaykrishnan Narayanan, Mehrdad Mahdavi
Abstract:
Activation steering offers a lightweight alternative to fine‑tuning for controlling large language models at inference time. While many existing methods implicitly optimize a log‑density‑ratio objective between desired and undesired activation distributions, they do so heuristically rather than deriving it from a principled optimization problem. Moreover, these methods produce query‑independent steering directions that can degrade performance on both in‑distribution and out‑of‑distribution (OOD) inputs. We introduce \textscCobras (Conditional Optimal Bridge for Riemannian Activation Steering), which addresses both limitations by casting activation steering as a Schrödinger Bridge on the residual‑stream hypersphere. This formulation yields, to our knowledge, the first principled derivation of the log‑density‑ratio steering objective from a well‑posed optimization problem. Solving the bridge via entropic optimal transport and extracting the probability flow ODE recovers the widely used density‑ratio gradient as a special case when the Sinkhorn potentials are uniform. Crucially, the Schrödinger potentials are evaluated at the current activation, making the resulting steering direction inherently query‑adaptive. Empirically, across four models and three alignment axes (helpfulness, truthfulness, and detoxification), \textscCobras consistently outperforms prior activation steering baselines while avoiding the OOD degradation commonly observed in existing methods. The code can be found at https://github.com/arshandalili/cobras.
Authors:Md Tanvir Islam, Sai Navaneet Peddapalli, Sangmoon Lee, Sangtae Ahn
Abstract:
Generative vision‑language‑action policies have advanced robot manipulation, but they often exhibit instability under noise, partial observability, and stochastic initial conditions. During extended rollouts, small velocity errors accumulate, degrading execution reliability. Existing diffusion and flow‑based policies typically assume homoscedastic residuals and lack explicit uncertainty modeling within action generation, limiting robustness during iterative rollout. We propose SUREFlow, a state‑space uncertainty‑aware residual flow matching framework built on a Mamba backbone. The method jointly predicts action velocities and input‑dependent residual uncertainty, enabling selective refinement of unreliable action dimensions without environment feedback while preserving computational efficiency. On LIBERO, SUREFlow achieves 92.5% average success rate (SR), outperforming the Mamba‑based MaIL by 34.2%. On LIBERO‑PRO, it attains around 49% SR using only 179M parameters, achieving performance comparable to large VLAs with 3‑7B parameters. SUREFlow source code is available on: https://github.com/tanvirnwu/SUREFlow
Authors:Filip Pawlicki, Marcel Kańduła, Marcin Pucek, Kamil Dobies
Abstract:
Recent Video Super‑Resolution (VSR) methods rely heavily on transformers and explicit optical flow, creating computational overhead and custom operations that hinder deployment on hardware accelerators like TensorRT. To address this, we introduce NanoVSR, a scalable, fully convolutional architecture designed for resource‑constrained edge devices. Using structural reparameterization, NanoVSR collapses into standard convolutions during inference, ensuring seamless hardware compatibility and negligible runtime overhead. Furthermore, despite lacking explicit motion compensation, it maintains competitive restoration quality by implicitly learning spatio‑temporal alignments through progressive training. Evaluated on the REDS4 benchmark, NanoVSR demonstrates an exceptional balance between accuracy and computational efficiency, significantly improving the trade‑off for compact architectures. Our NanoVSR‑644k baseline yields 28.64 dB PSNR while delivering 27.2 FPS on the NVIDIA Jetson Orin NX 16GB (25W), offering massive speed gains over heavier models. The scaled NanoVSR‑1.7M variant reaches 29.15 dB with a throughput of 19.58 FPS, providing superior, edge‑optimized upscaling. Code is available at https://github.com/filippawlicki/nanovsr.
Authors:Aaron Maurice Berman, Shantanu Dave
Abstract:
We introduce Grassmannian splatting, a dynamic scene representation whose primitives are Gaussians supported on 3‑planes in spacetime \R^4: generically, spatial 2‑planes in uniform translation along their normals. Each primitive carries a unit normal n \in \mathbb S^3/\\pm 1\ \cong \mathrmGr(3,4) and an unconstrained factor L \in \mathbb R^4 × 3, with covariance \[ Σ_4\mathrmD = (P_n L)(P_n L)^T, \qquad P_n = I ‑ n n^T. \] For generic L and n \neq \pm e_0, conditioning on time returns a rank‑2 surfel at every frame. The normal of the disk and its velocity along that normal are read off from n; the disk shape and the tangential drift of its center are set by L. Existing native 4D Gaussian splatting methods [\itYang et. al. 2023,Duan et. al. 2024] slice full‑rank spacetime covariances, so their per‑frame primitive is a volumetric ellipsoid; since conditioning lowers rank by exactly one, a rank‑2 surfel in the slice requires a rank‑3 spacetime covariance, and the parameterization above realizes exactly these. The motion model is closed form, i.e. no deformation field is learned, and no custom CUDA is required: the conditioned disk feeds a standard 3DGS rasterizer through its precomputed‑covariance interface. A soft clamp in the Schur denominator regularizes the static orientation and continuously bridges rank‑3 static and rank‑2 dynamic behavior, so static and moving primitives form a single continuous family. On the 17 HyperNeRF scenes of MonoDyGauBench, training is fastest among all compared methods (4.9 to 5.6 times faster than the strongest quality baselines), while ranking second in PSNR, MS‑SSIM, and LPIPS. Code: https://github.com/PaulCelanCoding/grassmannian‑splatting
Authors:Kefan Song, Yanjun Qi
Abstract:
Autonomous CLI agents can now execute hundreds of actions across multi‑hour sessions: writing code, executing shell commands, browsing the web, and managing cloud infrastructure, all with minimal human oversight. Does greater autonomy invite greater risk? We introduce ANCHOR, an automated auditing framework that stress‑tests CLI agents on illegal tasks grounded in public US court cases. ANCHOR deploys an auditor agent fine‑tuned on dark personality data using supervised and reinforcement fine tuning. This auditor roleplays persistent malicious users who decompose tasks, reframe requests upon refusal, and adapt strategies across multi‑turn interactions. Evaluating frontier CLI agents, we find that while they often refuse illegal tasks when prompted directly, compliance reaches 100% under persistent malicious interaction. When agents comply, they frequently exceed user requests, autonomously building infrastructure for large‑scale harm, including catastrophic risk scenarios such as large‑scale financial fraud and bioweapon development. These findings demonstrate that current alignment techniques are insufficient for autonomous agents and underscore the need for safety evaluations against persistent, adaptive malicious users. We release ANCHOR at https://github.com/garified/anchor
Authors:Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi, Ignacio Meza De la Jara, Sebastian Doerrich, Marco Lents, Christian Ledig
Abstract:
Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black‑box feature extractors, where only the last layer is considered for downstream tasks. We challenge this convention by introducing the notion of recoverability: the capacity of intermediate representations to correct last‑layer failures. By evaluating independent classification probes at every model depth across 16 datasets, we observe that intermediate probes correctly classify 18% to 76% of samples that the last‑layer probe misclassifies. We show that these gains are not primarily driven by predictive diversity, but by a redundancy‑correctness correspondence, where the internal hierarchy acts as a series of stable, redundant probes of a shared discriminative signal. While established horizontal ensemble strategies (i.e., across multiple models) can improve performance, they incur high computational cost and ignore this vertical signal within a single model. To bridge this gap, we propose VFusion, a principled vertical aggregation strategy employing a learnable mapping into a low‑dimensional latent space that synthesizes features across the internal ViT hierarchy. VFusion substantially outperforms established aggregation baselines in both in‑distribution and out‑of‑distribution settings, notably closing 45% of the accuracy gap between the best individual layer and a theoretical oracle performance. Our gains consistently generalize across model sizes and pre‑training regimes, confirming that VFusion offers a robust and efficient alternative to horizontal ensemble methods. The code is available at https://github.com/francescodisalvo05/vit‑vertical‑fusion.
Authors:Andrei Kuzmenko, Alexandr Maximenko, Aleksandr Kutsakov, Georgii Gospodinov, Dmitrii Bolotov, Oleg Kutuzov, Pavel Bogomolov, Fyodor Minkin
Abstract:
Despite recent scaling successes, multilingual ASR performance remains highly uneven, with long‑tail languages suffering from severe data scarcity. This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages (Kazakh, Kyrgyz, Uzbek). We present GigaAM Multilingual, a Conformer encoder pre‑trained on 2M hours of audio using a HuBERT‑style objective. Crucially, we introduce a cluster‑level data balancing strategy during pre‑training and a domain‑aware sampling method during fine‑tuning to mitigate head‑language dominance. In controlled comparisons, our approach outperforms strong open pretrained encoders (Whisper Large v3, Omnilingual‑1B) on target languages, achieving significant gains on spontaneous speech while maintaining efficiency. We release the foundation encoder and ASR model, offering a proven recipe for effective multilingual adaptation under realistic data imbalance.
Authors:Luca Cazzola, Giulia Martinelli, Nicola Conci
Abstract:
Motion blending in character animation enables the synthesis of new motions by interpolating between existing examples. Current methods are typically restricted to fixed skeleton topologies, requiring identical or near‑identical skeletal structures across characters. We present a novel framework for motion blending across heterogeneous skeletons. The proposed architecture combines a semantic encoder, which extracts per‑frame latent representations of the motion state, with a diffusion‑based decoder, which reconstructs character‑specific motion conditioned on this latent code. At inference, blended motions are obtained by interpolating the latent representations of two input motions. We train and evaluate the method on the Truebones Zoo dataset using motions defined on both same and distinct skeleton topologies, demonstrating the ability to achieve smooth and plausible blending in a variety of scenarios.
Authors:Rushuai Yang, Zhuo Han, Houlin Li, Hecheng Wang, Zhichao Wu, Rui Zhang, Zhaowei Zhang, Zihong Chen, Xiaohan Yan, Chiming Liu, Yi Chen, Wei Shan, Maoqing Yao
Abstract:
Flow‑matching policies have emerged as an effective policy parameterization for robot learning. They iteratively generate actions from noise, enabling highly expressive modeling of complex and multimodal action distributions. However, prior works observed that scaling these policies with value‑gradient reinforcement learning (RL) often leads to training instability. Existing methods attribute this instability to iterative generation and therefore avoid end‑to‑end value‑gradient optimization by sacrificing iterative generation, high expressiveness, or value‑gradient optimization. Contrary to prior belief, we show the instability does not stem from iterative generation itself, but from the vanilla sampling strategy originally designed for behavior cloning, which becomes brittle under value‑gradient RL. Motivated by this insight, we propose VINE, an RL‑oriented sampling method that enables stable end‑to‑end value‑gradient optimization for flow‑matching policies. Instead of following a single flow trajectory, VINE reconstructs a new interpolation state at every denoising step, creating a stable differentiable path for value‑gradient propagation while remaining compatible with the original flow‑matching denoising process. As a result, VINE preserves the expressiveness and iterative generation of flow‑matching without sacrificing end‑to‑end value‑gradient optimization. Despite performing end‑to‑end backpropagation through all ten denoising steps, VINE achieves stable policy improvement and consistently outperforms state‑of‑the‑art RL methods on the OGBench offline RL benchmark and real‑world robotic manipulation task. Videos are available on our website: https://agibottech.github.io/vine.
Authors:Giang Nguyen, Raghav Mehta, Emma A. M. Stanley, Tian Xia, Thi Hao Nguyen, Hieu Pham, Ben Glocker
Abstract:
Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear. We benchmark 15 foundation‑model backbones across breast density, BI‑RADS severity, and cancer status using a unified frozen‑backbone linear‑probe protocol, training on 3 source datasets and evaluating on 12 task‑compatible out‑of‑distribution (OOD) datasets after label harmonization. Mammography‑specific vision‑language models (Mammo‑FM and MaMA) provide the strongest mean OOD performance, but robustness is not explained by mammography exposure alone. DINOv3 remains a competitive vision‑only baseline, and mammography‑adapted pretraining does not consistently improve generalization. Dataset‑level analysis further shows that even leading models show heterogeneous performance across datasets. Feature‑space inspection reveals that useful representations can preserve clinical signal while retaining dataset and acquisition structure. These findings highlight dataset‑level OOD evaluation as a central criterion for assessing mammography representations. Our code is publicly available: https://github.com/biomedia‑mira/mammo‑ood.
Authors:Yash Shah, Omar Todd, Philipp Seeböck, Georg Langs, Ben Glocker, Raghav Mehta
Abstract:
The automatic detection and classification of cardiovascular disease (CVD) from computed tomography (CT) images plays an important role in clinical practice. Recently, a hybrid pipeline (GRC‑Net) for CVD classification was proposed, which leverages a deep‑learning‑based segmentation and registration method to extract radiomic and geometric features. However, GRC‑Net relies on a deterministic segmentation mask, without considering the inherent ambiguity associated with cardiac anatomy. In this paper, we propose GRC‑ProbNet, which takes advantage of a deep ensemble to produce multiple segmentation masks for a given input. From these masks, we extract multiple uncertainty features. We analyze these uncertainty features for both their correlation with segmentation error and their propagation effects on downstream CVD classification performance. Our experiments on the publicly available MM‑WHS and ASOCA datasets show that the uncertainty measure that best reflects segmentation quality is not necessarily the one that provides the strongest signal for downstream CVD classification. Overall, our results demonstrate that GRC‑ProbNet utilizing uncertainty features substantially improves CVD classification AUROC (92.92\) compared to the baseline GRC‑Net model (91.25%). Our code is publicly available: https://github.com/biomedia‑mira/GRC‑ProbNet.
Authors:Jiayi Tian, Shiao Liu, Yuting Xu, Jia Lu, Zihao Guan, Honglin Han, Di Yang, Minqi Gu, Yifei Qian, Tianlin Zhang, Yanqing Zhu, Zeqian Ye, Menglin Yang, Fei Wang, Xu Hu, Xiuxian Li, Wei Zhang, Shihui Su, Yiyan Ji, Jingbo Wang, Ziteng Feng, Jiaheng Liu, Zhaoxiang Zhang, Xiaolong Wu, Mingyang Yin, Zedong Chu, Mu Xu
Abstract:
Recent VLM and VLA systems have improved robotic perception and action prediction, yet long‑horizon embodied agents still require a general runtime layer for reasoning, memory, tool use, verification, and cross‑embodiment execution. We present ABot‑AgentOS, a general robotic Agent Operating System that sits above low‑level controllers and provides a deliberative agent layer for scene‑conditioned planning, context‑isolated skill execution, multi‑stage verification, multi‑modal memory, and edge‑cloud collaboration. To evaluate such systems, we introduce EmbodiedWorldBench, an executable benchmark with 16 indoor, outdoor, and hybrid scenes, four difficulty levels, and over 200 tasks involving navigation, object search, NPC dialogue, dynamic events, and trace‑grounded scoring. ABot‑AgentOS further introduces Universal Multi‑modal Graph Memory, a persistent source‑grounded substrate that converts dialogue, visual observations, spatial context, temporal relations, and task traces into typed nodes and edges. A failure‑driven self‑evolution loop converts diagnosed memory failures into gated runtime evo‑assets that are promoted only to later evaluation splits, preventing current‑split ground‑truth leakage while enabling continual improvement. On an initial EmbodiedWorldBench subset, ABot‑AgentOS improves over a single‑controller baseline in both task success and goal completion. Across memory benchmarks, ABot‑AgentOS Static achieves 87.5 on LoCoMo, 59.9 on OpenEQA EM‑EQA, 88.6 on Mem‑Gallery, and 76.5 Acc@All on NExT‑QA; self‑evolution further improves LoCoMo to 88.7, OpenEQA to 60.4, and Mem‑Gallery to 89.0. These results suggest that a general Agent OS layer can improve long‑horizon embodied execution while providing persistent, auditable memory for continual interaction.
Authors:Shihao Yuan, Yuanze Li, Ruyi Zhang, Ming Liu, Wangmeng Zuo
Abstract:
Despite the advancements of Large Multimodal Models (LMMs) in RGB vision, their ability to generalize to unseen visual modalities remains a largely unexplored challenge. We argue that different visual modalities are merely distinct samplings of the same physical world. Therefore, effective generalization requires models to possess both modality‑agnostic perception of scene semantics and the adaptability to modality‑specific characteristics. To achieve this, we propose a training framework, VVM‑Tuning, to equip LMMs with these capabilities through modality synthesis and modality contexts. Specifically, we synthesize diverse appearance‑varied images from RGB scenes, training the model to disentangle invariant semantics from varying visual appearances, and align these appearances with language for visual concepts decoupled from modalities. We then introduce modality contexts in the prompt and use instruction tuning to assist the model in mapping these appearance variations back to modality‑related attributes, enabling zero‑shot adaptation to unseen modalities during inference. To facilitate research in this direction, we introduce VVM‑Bench, a comprehensive benchmark featuring 6 real and synthetic modalities to evaluate semantic perception and modality understanding. Experiments demonstrate that, via our training on synthetic modalities, 5 tested models exhibit consistent improvements on both real‑world and novel synthetic modalities without in‑modality training. Source code and data will be publicly available at https://github.com/Hunter‑Will/VVM‑Tuning.
Authors:Jiakang Yu, Yixuan Chai, Tianci Wang, Rihui Jin, Guangkai Xu, Hongtao Deng, Xun Zhu, Wang Gao, Xinrun Guo, Haipang Wu
Abstract:
Generative image editing models struggle with structured statistical charts when data modifications require geometric synchronization. We formalize this task as Visuo‑Logical Cascading Editing (VLCE). However, existing methods remain confined to localized text substitutions and struggle with dependency‑aware cascading updates. To systematically evaluate this capability, we introduce ChartSync, an expert‑validated benchmark constructed via a programmatic rendering pipeline that guarantees deterministic visuo‑logical coupling for the ground truth. ChartSync comprises 870 triplets across 9 chart categories and 4 task types, including 235 geometry‑coupled VLCE instances that specifically test cascading text‑to‑geometry synchronization. We further evaluate these instances via a two‑tier framework combining objective visual metrics with a vision‑language model judge paradigm to assess low‑level fidelity alongside multimodal comprehension and reasoning. Evaluating 14 image editing models and one code‑mediated pipeline reveals a nuanced capability gap: most open‑source models suffer severe drops in geometric synchronization, while only two frontier proprietary models show emerging VLCE capability, with their residual errors mainly involving semantic isolation and background corruption. Our detailed error analysis deconstructs these failure paradigms to identify core meta‑abilities for guiding future multimodal architectures. The ChartSync dataset and code are publicly released at https://github.com/kaka‑yjk/ChartSyncCodebase.
Authors:Kaiying Yan, Luoyi Sun, Xiao Zhou, Weidi Xie
Abstract:
Recent advances in large‑scale multimodal models have drivenremarkable progress in vision‑language tasks; however, comprehensiveomni‑modal understanding remains under‑explored, largely due to thescarcity of datasets with rich, explicitly aligned auditory cues. To bridgethis gap, we present AVDC (Audio‑Visual Decoupled Captions), a large‑scaledataset designed to disentangle visual and auditory semantics. Specifi‑cally, we propose an automated pipeline that leverages off‑the‑shelf mod‑els to annotate videos with tripartite captions: visual‑only (V), audio‑only (A), and joint audio‑visual (AV). This decoupled structure explic‑itly captures both modality‑specific nuances and complex cross‑modalinteractions. Building upon this, we introduce AVDC‑QA‑CoT, a Chain‑of‑Thought augmented question‑answering dataset to foster audio‑visualreasoning. To fully exploit these resources, we employ a two‑stage train‑ing paradigm: omni‑modal caption generation pre‑training on AVDC, fol‑lowed by instruction tuning on AVDC‑QA‑CoT. Extensive experiments acrossdiverse downstream tasks, spanning video captioning, audio‑centric anal‑ysis, and omni‑modal benchmarks, demonstrate consistent and signifi‑cant performance gains, showing the efficacy of our proposed datasetsand training strategy in advancing omni‑modal perception. Code anddataset are related on https://radiant0726.github.io/AVDC‑web/.
Authors:Xuankun Rong, Wenke Huang, Bo Du, Dacheng Tao, Mang Ye
Abstract:
As large language models (LLMs) are increasingly used in decision support, it is important to understand whether their choices under uncertainty exhibit stable and interpretable behavioural regularities. Human decision‑making combines relatively persistent risk preferences with context‑dependent adjustment, yet it remains unclear whether analogous behavioural structure can be observed in LLM‑based decision systems. Here we examine this question using a controlled multi‑model framework based on no‑limit Texas Hold'em, where behaviour is quantified by Participation, measuring voluntary engagement in uncertain opportunities, and Proactiveness, measuring pre‑flop risk escalation. Across homogeneous self‑play and heterogeneous mixed‑model interactions, frontier LLMs exhibit stable, model‑specific risk profiles, forming a spectrum from conservative to aggressive decision styles. These profiles remain largely robust under changing opponent composition, while the most conservative and most aggressive models diverge further in mixed settings. Under global risk pressure and personal resource constraint, models adapt in structured but heterogeneous ways, ranging from broad behavioural contraction to selective de‑escalation and near‑invariant behaviour. These findings suggest that LLMs differ not only in baseline risk disposition, but also in the risk signals they respond to and the flexibility with which they adjust, providing a behavioural basis for auditing risk‑sensitive decision‑making in interactive settings. Our code is publicly available at: https://github.com/XuankunRong/AgentTexasPoker.
Authors:Zhiyan Zhang, Peipei Song, Jinpeng Hu, Jingyang Jia, Xun Yang, Xiaojun Chang
Abstract:
Video emotion analysis is typically framed as a static classification problem, treating each clip as an independent labeled unit. However, such a formulation overlooks a key psychological fact: emotions change as a result of cumulative reactions to consecutive causal events. To bridge this gap, we introduce Dynamic Affective Reasoning, the first large‑scale benchmark for viewer‑centric affect transitions and causal reasoning over consecutive video events. DAR contains 15,087 videos and 36,908 event‑aligned affective segments annotated with 27 emotion categories. Unlike existing video‑based emotion datasets, DAR presents a new viewer‑centric perspective on fine‑grained emotional expressions and transitions, and provides dense, temporally grounded, and causally explicit reasoning chains. Based on DAR, we formally define three challenging tasks: affective segmentation, fine‑grained emotion classification, and affective reasoning. Complementing this benchmark, we propose DAR‑R1, a two‑stage framework that combines supervised fine‑tuning with Group Relative Policy Optimization. Experiments across 10+ MLLMs show that DAR‑R1 sets a new state‑of‑the‑art for dynamic affective reasoning, in terms of both emotional localization and affective reasoning. Project page: https://github.com/Zhang‑Zhiyan/DAR.
Authors:Nipun Misra, Vikranth Udandarao, Aanchal Gupta, Yogender Kumar, Manuj Mukherjee, Raghava Mutharaju
Abstract:
Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance. Existing evaluation practices rely heavily on task‑specific metrics or small‑scale manual verification, offering limited insight into the structural and semantic fidelity of extracted graphs. We propose a novel, interpretable metric for intrinsic KG quality assessment that measures how closely an automatically extracted graph approximates an "ideal" graph capturing the key noun phrases, predicate relations, and basic linguistic phenomena such as negation expressed in the source text. Our framework integrates two complementary components: (1) an entity‑level assessment that evaluates completeness, resolution quality, and connectivity, and (2) a relation‑level assessment that judges predicate preservation and multiplicity using lexical similarity, dependency‑parse alignment, and light‑weight negation handling to ensure semantic faithfulness. We evaluate our metric across multiple state‑of‑the‑art triple extraction systems and datasets, including WebNLG, TinyButMighty, and BenchIE, demonstrating that it reliably identifies omissions, redundancy, and structural deviations that existing metrics overlook. Our work offers a scalable, model‑agnostic, and interpretable framework for comparing automated KG construction methods and provides a foundation for standardised evaluation. We further validate the metric through an ablation study isolating noun and verb components, and a downstream evaluation showing that KGCQual scores correlate significantly with link prediction performance on the same extracted KGs. The code repository is available at https://github.com/kracr/kg‑quality‑metric.
Authors:Jui-Te Huang, Ruoyang Xu, Michael Kaess
Abstract:
Robust probabilistic mapping is essential for autonomous robotic systems operating in challenging environments. While traditional sensors fail in adverse conditions such as smoke and fog, millimeter wave (mmWave) radar sensors offer reliable sensing in such conditions. However, creating accurate probabilistic maps from radar data presents significant challenges due to the inherently sparse and noisy characteristics of radio wave measurements and signal processing steps. In an attempt to address these issues, we establish a complete pipeline from raw radar signals to probabilistic occupancy maps, incorporating Synthetic Aperture Radar processing followed by a probabilistic modeling step. We conduct extensive validation across indoor environments, comparing our approach against different signal processing and probabilistic modeling approaches. We also evaluate mapping quality through downstream path planning performance analysis. Furthermore, we investigate the impact of key parameters and antenna array configuration on mapping performance. The experimental results demonstrate both the effectiveness and limitations of SAR‑based probabilistic mapping for real‑world robotic deployment. To facilitate future research and broader adoption, we contribute an open‑source cascaded mmWave radar dataset with an accompanying GPU‑accelerated signal processing pipeline available at https://github.com/rpl‑cmu/rpm.
Authors:Li Guo, Anas M. Tahir, Z. Jane Wang
Abstract:
Automated chest X‑ray report generation has recently benefited from reinforcement learning (RL) and large language models. However, RL training often suffers from instability or limited exploration due to fixed Kullback‑Leibler (KL) regularization and a static reference policy that accumulates KL pressure over time. We propose Response‑Weighted and Validation‑Anchored Policy Optimization (REVA‑PO), a RL framework that stabilizes long‑term training via Response‑Weighted Regularization (RER) and Validation‑Anchored Policy Reset (VAPR). RER dynamically adjusts per‑response KL weights based on advantage and reference‑policy entropy, relaxing constraints for high‑quality responses while tightening them for low‑quality ones. Complementarily, VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure to expand the viable exploration space. To ensure a robust starting point, we employ a three‑stage pipeline consisting of warm‑up training, classifier‑guided supervised fine‑tuning, and RL. Extensive evaluations on MIMIC‑CXR and IU‑Xray demonstrate that REVA‑PO sets new state‑of‑the‑art benchmarks in both linguistic quality and clinical accuracy. Notably, BLEU‑4 improves by 5.1% on MIMIC‑CXR and 3.6% on IU‑Xray, while CheXpert F1 and RadGraph F1 scores increase by 4.5% and 12.8%, respectively, over prior leading methods. The code is publicly available at https://github.com/LiGuo12/REVA_PO/.
Authors:Yuang Meng, Chenyang Wu, Xianshun Liu, Chun-Le Guo, Zichen Liang, Lina Lei, Jie Liang, Hui Zeng, Chongyi Li, Lei Zhang
Abstract:
Existing optical flow methods broadly follow two paradigms: iterative optimization and diffusion‑based estimation. Iterative methods, exemplified by RAFT, achieve high accuracy through recurrent refinement, but remain challenged by large displacements and complex motion. Diffusion‑based methods introduce generative modeling and show promise in such ambiguous regions. However, existing diffusion models usually denoise the entire dense flow field from Gaussian noise, including simple regions where reliable motion can already be estimated by a lightweight network. This increases the denoising burden and may cause slow convergence and unstable training. To address this issue, we introduce FlowPainter, a diffusion‑based optical flow framework that reformulates dense‑flow generation as confidence‑guided soft inpainting. FlowPainter employs a lightweight confidence‑aware network to predict a rough flow and a pixel‑wise confidence mask, distinguishing reliable simple regions from uncertain hard regions. The resulting simple‑flow prior is used for confidence‑based initialization and further injected into iterative denoising through confidence‑gated residual guidance. With dynamically decaying guidance strength, FlowPainter stabilizes early denoising while preserving the flexibility of the diffusion model for late‑stage detail refinement. Extensive experiments on public benchmarks, including Sintel, KITTI, and Spring, show that FlowPainter achieves strong accuracy under comparable training settings and converges more efficiently than existing diffusion‑based optical flow methods, with notable gains on challenging benchmark splits. Our approach offers a practical way to integrate reliable discriminative priors with diffusion‑based refinement for optical flow estimation. Our code is publicly available at https://github.com/mya012/FlowPainter.
Authors:Junhui She, Fei Wang, Kun Li, Yiqi Nie, Yuxin Liu, Zhangling Duan, Xun Yang
Abstract:
Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi‑branch methods require extra supervision and annotations, while streamlined designs prioritize low‑level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross‑modal architecture that leverages a Large Vision‑Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze‑aligned textual cues. We design a transformer‑based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in‑/out‑of‑frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross‑dataset generalisation without extra fine‑tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation.
Authors:Xianzhi Ma, Shujun Wang, Xiaohan Li, Hao Liu, Changhua Pei, Jianhui li
Abstract:
Ultra‑High‑Resolution (UHR) remote sensing image understanding requires Vision‑Language Models (VLMs) to capture both the global scene layout and sparse yet task‑critical local details under limited computational budgets. Existing methods mainly follow two paradigms. One is passive perception, which relies on resolution expansion or token compression and may therefore discard fine‑grained details. The other is active perception, which depends on multi‑round zooming and search, but suffers from high latency, contextual fragmentation, and error accumulation. We argue that a more effective path toward UHR understanding lies not in accessing more, but in organizing better. To this end, we propose WeaveEarth, a training‑free framework that reformulates UHR understanding as a problem of structured evidence construction and reasoning under global context constraints. Specifically, WeaveEarth first employs Global‑Aware Evidence Construction to select a compact, low‑redundancy, and spatially complementary Minimal Support Evidence Set. It then introduces Structured Evidence Reasoning, which weaves local evidence, spatial metadata, and relative topology into a unified reasoning interface, thereby enhancing the VLM's ability to perform global‑local joint reasoning. Extensive experiments show that WeaveEarth consistently outperforms strong baselines and existing UHR methods across multiple UHR remote sensing benchmarks and multiple frozen VLM backbones. Code is available at https://github.com/XianZhi‑Ma/WeaveEarth.
Authors:Chigozirim Ifebi, Brent Kong, Ayushi Mehrotra
Abstract:
Safety alignment in large language models remains brittle across languages: prompts reliably refused in English can elicit harmful compliance in non‑English and low‑resource settings. We introduce \textscMinionese, a multilingual jailbreak benchmark spanning 18 languages, 4 resource tiers, and 4 perturbation types (standard translation, code‑switching, transliteration, and translationese), paired with a geometric mechanistic analysis of refusal failure across language tiers. We show that each attack type produces a distinct vulnerability profile: transliteration vulnerability is mediated by script identity, code‑switching maintains effectiveness through the lowest‑resource tier, and a sharp safety regime transition between Tiers 2 and 3 is consistent across all models. Mechanistically, low‑resource jailbreaks succeed by routing harmful content through a geometrically misaligned subspace that projects insufficiently onto the refusal directions, leaving the refusal mechanism intact but untriggered. These findings show that English‑only safety evaluations are insufficient; they require accounting for script family, perturbation type, and per‑language alignment coverage. The benchmark and analysis code is at https://github.com/Brentkong/Minionese‑Comprehensive‑Benchmark‑and‑Mechanistic‑Study‑of‑Multilingual‑LLM‑Safety.git.
Authors:Ryota Sato, Eli Silverstein
Abstract:
WaveNet‑style convolutional networks emulate tube amplifiers and distortion pedals with high fidelity, but their computational cost has confined them to desktops or dedicated DSP hardware. We present a sparse‑enabled WaveNet inference engine for iOS that runs heavily pruned neural guitar amplifier models in real time on iPhones. Aggressive iterative magnitude pruning removes 90% of the network weights with no perceptible loss in quality. A custom sparse C++ engine turns this sparsity directly into compute savings, sustaining low‑latency real‑time operation on a CPU‑only iPhone implementation where the dense model cannot. On‑device output matches the trained model to within int16 quantization error. At the demonstration, visitors will play a guitar through the app on iPhone hardware and A/B the on‑device pruned model against the physical pedal it emulates. Source code and audio examples are available at https://github.com/ryos17/wavenet‑imp.
Authors:Zhonghua Yi, Hao Shi, Qi Jiang, Yufan Zhang, Kailun Yang, Kaiwei Wang
Abstract:
Building pixel‑level correspondence between event and image data is a fundamental task for multi‑sensor systems. However, existing cross‑modal matching methods are largely restricted by their reliance on either matching labels or strictly aligned hardware, which limits them to unlabeled and unconstrained real‑world scenarios where neither matching ground truth nor prior sensor relationships are available. To address this, we propose a novel two‑stage training paradigm. First, we leverage large‑scale data to perform label‑agnostic distillation pretraining, upgrading optimization objectives with distribution‑based and contrastive losses to learn highly generalizable representations. Second, to tackle unlabeled and unconstrained downstream data, we introduce an epipolar‑guided self‑distillation framework. By utilizing consistency verification to isolate robust matches and incorporating geometric confidence derived from an external epipolar prior, our model can effectively self‑evolve directly on target domains without any supervision. Furthermore, we introduce a rigorous cross‑modal evaluation benchmark based on TUM‑VIE, featuring physically separated cameras with distinct intrinsic parameters and resolutions. Extensive experiments demonstrate that our proposed method achieves state‑of‑the‑art performance on both MVSEC and TUM‑VIE pose estimation tasks. The source code and benchmark will be made publicly available at https://github.com/ZhonghuaYi/nexus2‑official.
Authors:Shunsuke Yokokawa, Hironori Kasahara
Abstract:
Bird's‑eye‑view (BEV) perception is a core component of camera‑based 3D understanding in autonomous driving, where view transformation (VT) maps multi‑camera image features into a unified BEV representation. Sampling‑based view transformation (Sampling‑VT) is attractive because it supports dense and continuous BEV aggregation for high‑resolution and long‑range perception. Its deployment bottleneck, however, is systems‑level: standard tensorized implementations of Sampling‑VT ‑‑ which we refer to as Tensorized Sampling‑VT ‑‑ explicitly materialize large height‑dependent intermediate tensors, causing memory and latency costs that scale poorly with vertical resolution and the number of cameras. We revisit Tensorized Sampling‑VT from an operator‑execution perspective and show that it follows a gather‑reduction pattern: each BEV query independently accumulates contributions across cameras and height bins, enabling thread‑local accumulation with on‑the‑fly recomputation that eliminates the need to materialize height‑ and camera‑dependent intermediates. Based on this insight, we propose FlashBEV, a fully fused and IO‑aware execution strategy mathematically equivalent to Tensorized Sampling‑VT (same operator output) while substantially reducing global memory traffic and kernel‑launch overhead. Experiments show that FlashBEV achieves more than an order of magnitude lower peak GPU memory and significant inference‑latency speedups, with memory effectively independent of the number of height bins, reducing the operator's peak memory to O(BCXY) (output only). This unlocks higher BEV range/resolution and vertical discretization within fixed deployment budgets on memory‑constrained devices. Our contribution is an execution redesign ‑‑ same math, different execution ‑‑ that removes a key scalability barrier for deployment‑ready Sampling‑VT. Code available at https://github.com/yokosyun/FlashBEV
Authors:Ripon Chandra Malo, Shatabdi Roy, Tong Qiu
Abstract:
Academic and project maps are often produced through a fragmented workflow: researchers locate boundaries, manage shapefiles, join tabular data, assemble locator insets, add cartographic decorations, and export figures through desktop GIS or multi‑package Python scripts. This creates an accessibility barrier for non‑GIS users and a reproducibility problem when data sources, styling choices, and manual edits are not captured in executable form. We present AcadGIS, a free and open‑source Python package that creates publication‑oriented research maps from high‑level commands under one namespace, import acadgis as agis. AcadGIS provides place‑name boundary access, automated study‑area locator layouts, thematic cartography, raster and vector layers, curated Earth‑observation products, terrain and hydrology context, and configurable PNG, PDF, and SVG export without requiring desktop GIS expertise or hand‑managed shapefiles. Its design combines one‑import access to the scientific‑Python stack, publication‑oriented defaults with progressive control, local caching, source attribution, and figure specifications based on code, named data, and a pinned package version. Through three representative use cases, we demonstrate how common paper, thesis, and project maps can be expressed as compact, inspectable scripts. Source code: https://github.com/riponcm/AcadGIS.
Authors:Mingyang Yao, Zhaoxiang Feng
Abstract:
Self‑supervised learning for symbolic music has advanced largely through token‑level pretraining, but such representations remain tied to tokenizer‑specific sequences and often provide time‑span‑level embeddings only indirectly. In this paper, we propose ARIMA, a reconstruction‑grounded latent predictive framework for symbolic music that learns compact window‑based representations directly from data. ARIMA encodes each fixed‑duration window into a continuous latent representation, trains a causal predictor with contrastive next‑latent prediction, and grounds the encoder through structured reconstruction of music elements. This design preserves local musical details while modeling temporal progression across windows. We evaluate ARIMA on downstream tasks spanning various levels of music understanding. Results show that ARIMA is particularly efficient and effective on tasks involving harmonic, timing, and cross‑performance retrieval, while remaining competitive with much larger baselines on other tasks. Ablations further show that next‑latent prediction is essential for temporally integrated representations, and that structured reconstruction stabilizes latent learning without requiring explicit variance regularization. The code is at https://github.com/AndyWeasley2004/symbolic_music_wm.
Authors:Yang You, Yi Du, Cole Harrison, Leonidas Guibas
Abstract:
Object pose estimation is a fundamental problem in 3D vision. Although recent state‑of‑the‑art approaches achieve strong performance, they often overfit to existing benchmarks and exhibit limited generalization to novel categories and unseen scenes. We propose UniPose9D, a category‑agnostic foundation model for 9D object pose estimation: given an instance mask/ROI and either an RGB‑D observation or an RGB image with predicted depth, the model estimates rotation, translation, and metric size without category labels, CAD models, mean‑shape priors, or reference views. Specifically, UniPose9D samples point pairs from the observed object geometry and uses DINOv2 and PointNet features to predict NOCS coordinates for each pair. To improve accuracy, we introduce a point‑pair‑based RANSAC N‑hop Kabsch‑‑Umeyama algorithm with an adaptive threshold. We further employ flow matching to address symmetric ambiguities and construct a large‑scale training set by curating and aligning pose annotations from existing public datasets. Experiments across six datasets show that a single unified model can match or surpass specialist methods while generalizing to unseen objects and in‑the‑wild scenarios. Our code and model are available on https://github.com/qq456cvb/UniPose9D.
Authors:Afonso E. Carvalho, David Portugal, Paulo Peixoto
Abstract:
iG‑LIO is a tightly‑coupled LiDAR‑inertial odometry system fusing generalized‑ICP and point‑to‑plane constraints in an iterated error‑state Kalman filter over an incremental voxel map. We report an open‑source ROS 2 Jazzy port of the original ROS 1 implementation and, more importantly, the diagnosis of environment‑induced numerical failures that appear only after the port: a mechanically faithful migration ‑‑ estimation mathematics left unchanged ‑‑ compiled and ran, yet diverged with NaN internal values. Both causes trace to the modern ROS 2 toolchain, not the algorithm: a Quality‑of‑Service (QoS) mismatch that silently drops and reorders IMU samples, and an uninitialized parallel‑reduce accumulator arising from the oneTBB + Eigen combination shipped with current distributions. We further correct Ouster point‑field parsing to ensure correct point cloud undistortion with newer Ouster revisions, add Velodyne Velarray M1600 support, provide both a compile‑time‑gated Livox CustomMsg path and a driver‑free path for Livox sensors publishing standard PointCloud2 (e.g. Mid‑360), and expose the runtime via YAML. The result has been validated in an Ouster OS0 Rev7, an Ouster OS1 Rev 7, and a Livox MID‑360. This report is a citable reference for the port itself, not a claim on the underlying algorithm [1]. The ROS 2 port of iG‑LIO described in this document can be found at https://github.com/Forestry‑Robotics‑UC/ig_lio/tree/ros2‑jazzy.
Authors:Ivan Alejandro Montoya Sanchez, Anantaa Kotal, Aritran Piplai
Abstract:
Cybersecurity systems must adapt rapidly to emerging threats. However, labeled data for new threat categories is unavailable when those threats first appear. Generalized zero‑shot learning offers a natural solution by enabling recognition of unseen classes through auxiliary semantic knowledge rather than labeled examples. Large language models are particularly promising in this setting because they can convert unstructured CTI reports into semantic prototypes for emerging threats. However, applying language‑driven zero‑shot learning to cybersecurity is difficult due to strong semantic overlap between threat descriptions, heterogeneity between behavioral attributes and text, severe class imbalance, and open‑set conditions where unseen threats are unknown during training. We propose SMETA‑ZSL, that learns semantic prototypes from overlapping language descriptions through contrastive finetuning, aligns behavioral features through episodic meta‑learning and knowledge distillation, and performs adaptive routing for generalization across seen‑unseen classes. Across 7 benchmarks, SMETA‑ZSL delivers the strongest overall generalized zero‑shot performance under the strictest inductive setting, surpassing prior methods by 10.8 points on average, with gains up to 18.1 points. Github:https://github.com/Security‑And‑Intelligence‑Lab‑UTEP/SMETA‑ZSL
Authors:Jiarui Li, Joseph Brewington, Qingzhao Zhang, Z. Morley Mao
Abstract:
Gimbal‑stabilized visual tracking is critical for modern autonomous systems such as Unmanned Aerial Vehicles (UAVs). While prior work shows acoustic signals can disturb gimbal internals, the impact of such attacks on real‑world applications like UAV tracking and following remains underexplored. Existing demonstrations largely overlook practical challenges for real‑world attacks, such as object‑motion uncertainty and runtime latency. To bridge this gap, we present Banshee, the first physically realizable attack that induces target switching in UAV visual tracking systems by exploiting acoustic vulnerabilities in gimbal‑camera systems. Banshee generates carefully crafted acoustic waveforms that induce optimized adversarial gimbal oscillations, causing directionally biased camera‑view drifts that break inter‑frame target associations. Consequently, the onboard tracker is driven to switch from the original target to an attacker‑selected object with high probability, with occasional target loss. Banshee achieves a 93.6% success rate in simulation across two commercial gimbal systems and five trackers. Real‑world benchtop and in‑flight black‑box attacks against a commercial drone across varied scenarios show an overall 95.5% attack success rate. Our results reveal a practical cross‑domain vulnerability between acoustics and vision, highlighting the need for robust designs of gimbal systems and applications. Our code is available at: https://github.com/U1ltra/Banshee.
Authors:Sai Anirudh Katupilla, Shreeya Dasa Lakshminath
Abstract:
Inverse Reinforcement Learning recovers reward functions from expert demonstrations, but standard formulations assume that all demonstrations come from a single expert. When demonstrations are pooled from multiple experts with distinct preferences, parametric methods recover an averaged reward that fits no individual expert well. We implement Nonparametric Bayesian Inverse Reinforcement Learning with a Dirichlet Process prior over reward functions, allowing the number of latent reward types to be inferred jointly with the rewards themselves. Inference uses a collapsed Gibbs sampler combining a Chinese Restaurant Process update for cluster assignments with a Metropolis‑Hastings update for reward weights, and soft value iteration as the inner planning routine. We evaluate on a 10x10 ObjectWorld grid with two and three ground‑truth reward types. The serial sampler recovers K=2 with Adjusted Rand Index of 1.000, substantially outperforming a Maximum Entropy IRL baseline (ARI=0.000). Extension to K=3 shows that the sampler correctly identifies the number of clusters in all runs; assignment ARI of 0.48‑0.58 reflects behavioral overlap between expert types that persists across grid instantiations, revealing that reliable K=3 evaluation on ObjectWorld requires controlled object placement rather than random seeding. We further parallelize the sampler across CPU cores using Ray on HPC hardware, achieving a peak speedup of 4.79x at 8 workers, and characterize a throughput‑versus‑accuracy tradeoff arising from the consensus merge heuristic used during state aggregation. Code and a containerized environment are available at https://github.com/dasashreeya/np_bayes_irl.
Authors:Lusheng Zhang, Shien He, Tianxing Yan, Mengran Yu, Ziang Cui, Kai Zhao, Xiaojing Liu, Tianjiao Li
Abstract:
We present Index‑1.9B, a series of open small language models developed at Bilibili. The series comprises four models: Index‑1.9B‑Base, a foundation model with 1.9 billion non‑embedding parameters pre‑trained on 2.8 trillion predominantly Chinese and English tokens; Index‑1.9B‑Pure, a control variant trained with an identical recipe but with all instruction‑like data strictly filtered from the corpus; Index‑1.9B‑Chat, aligned from the base model with supervised fine‑tuning and direct preference optimization; and Index‑1.9B‑Character, which augments the chat model with retrieval‑augmented generation for few‑shot role‑playing customization. Pre‑training employs a Warmup‑Stable‑Decay learning‑rate schedule in which the concentration of curated data is raised substantially during the decay phase, together with a Norm‑Head output layer that stabilizes training under large learning rates. On a suite of standard benchmarks covering examination, reasoning, mathematics, and code, Index‑1.9B‑Base attains an average score of 64.92, competitive with or exceeding open models of several times its size. We further report controlled studies on model depth, learning‑rate magnitude and scheduling, the interaction between learning‑rate decay and data quality, and the effect of including instruction data during pre‑training, and we document an unexplained surge in benchmark performance midway through the constant‑learning‑rate phase. All models, together with evaluation code, are released at https://github.com/bilibili/Index‑1.9B.
Authors:Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak, Minh Nguyen, Ruining Deng, Mert R. Sabuncu
Abstract:
Multimodal medical models often degrade when inputs are missing, a common scenario in real‑world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over‑relies on a highly predictive modality and undertrains complementary sources, resulting in poor robustness under partial availability. While training‑time modality knockout improves missing‑modality robustness, existing approaches use static masking rates that cannot adapt to evolving modality utility during training. We introduce ShapKO (Shapley‑Adaptive Modality Knockout), a dynamic training strategy that learns modality‑specific knockout probabilities based on validation utility. ShapKO periodically evaluates performance across modality subsets, estimates modality importance via Shapley values, and updates masking probabilities to suppress dominant modalities more frequently. This adaptive process promotes complementary representations, while requiring no architectural modifications. We evaluate ShapKO on three datasets covering multitask clinical classification, survival prediction, and cancer detection. ShapKO consistently improves performance under modality absence and yields interpretable trajectories of learned masking behavior. Code is available at: https://github.com/sumona00/ShapKO
Authors:Weida Li, Zhuanghua Liu, Yaoliang Yu, Bryan Kian Hsiang Low
Abstract:
The Shapley value is a widely used concept in attribution problems, as it uniquely satisfies the axioms of linearity, consistency, equal treatment, and efficiency. Often, the inclusion AUC metric is used to evaluate the quality of player rankings, in order to identify positively participating players. However, it can be established that the Shapley value is not always reliable for this purpose. The core issue lies in its linearity: the Shapley value acts as a linear operator with an excessively large null space, which is likely to contain non‑negligible perturbations that remain indistinguishable to the operator. To address this limitation, we explore the design of nonlinear axiomatic attribution methods. Inspired by the least core, which is a popular nonlinear substitute for the Shapley value, we introduce a class of nonlinear attribution methods that retain the remaining necessary axioms. Each method yields a contribution vector that is the unique optimal solution to a minimization problem, which aims to approximate utility functions as faithfully as possible. In terms of the inclusion AUC metric, our experiments demonstrate the potential effectiveness of these methods compared to Shapley value variants that relax only the efficiency axiom. Our code is available at https://github.com/watml/nonlinear‑axiom.
Authors:Wenke Xia, Pei Ren, Wenbo Yu, Yizhuo Zhang, Jifan Li, Yixue Zhang, Yinuo Zhao, Qingyang Gao, Jianlong Fu, Jian Tang, Ji-Rong Wen, Zhengping Che, Di Hu
Abstract:
Offline‑to‑online reinforcement learning is promising for generalizable robotic manipulation, yet its full‑stack complexity obscures reproduction and diagnosis. Within such systems, value estimation plays a central role in prioritizing heterogeneous data for policy improvement. Despite its importance, the central question remains underexplored: how value‑function reliability shapes policy optimization in offline‑to‑online reinforcement learning. To answer this question, we propose Robo‑ValueRL, a unified framework that enables reliable value estimation and systematically traces its downstream effects on policy pretraining and online improvement. Concretely, Robo‑ValueRL learns a history‑conditioned value estimator and evaluates its reliability through global‑progress and local‑preference metrics. These resulting value estimates are propagated into quality‑conditioned consistency‑policy pretraining and a residual adaptation module on online rollouts, providing a unified testbed for analyzing how value reliability shapes downstream policy performance. Across 240 hours of offline demonstrations and over 3,000 online rollout trajectories, our extensive experiments show that downstream performance is strongly associated with value reliability. Reliable value functions provide better action‑quality estimates, allowing value‑guided offline RL to scale more effectively than quality‑agnostic behavior cloning, and stabilize online improvement by prioritizing high‑quality rollout data. Integrating reliable value guidance through offline pretraining with online improvement, our system achieves 86% success on millimeter‑level precise chip insertion and 84% on generalizable block disassembly. We hope these findings highlight the importance of value‑guided data utilization for effective policy improvement from heterogeneous robotic experience.
Authors:Hongtao Liang, Xinyu Shao, Chenxu Wang, Yiyao Wan, Jiahuan Ji, Fangwei Ye, Fuhui Zhou, Qihui Wu
Abstract:
Under Global Navigation Satellite System (GNSS) denial, a UAV controller still needs a distance and heading command it can execute, making accurate metric last‑meter navigation essential. Dense pair‑geometry foundation models transfer relative structure well, yet the distance scale of their raw metric outputs remains poorly calibrated. Under the relative error metric of PairUAV, correcting only the average scale can still leave costly, distance‑dependent residuals near the goal. To address this scale mismatch, Range‑Aware Scale Recovery (RASR) separates a transferable scale‑recovery core from a protocol‑specific calibration module in a per‑pair system fixed at inference. The core compresses frozen Matching And Stereo 3D Reconstruction (MASt3R)‑style geometry into a compact descriptor and uses global calibration to recover the dominant metric signal. Range‑bucket residual correction and command‑grid alignment stay inside the calibration module, so they match the command format and evaluation protocol of PairUAV. On the UAVs in Multimedia 2026 PairUAV online evaluation, RASR reaches a total score of 0.003189. Under the PairUAV protocol, frozen pair geometry thus yields stable per‑pair distance and heading estimates, while every protocol‑specific adjustment stays confined to a calibration module fixed before inference. Code and materials are available at https://github.com/lht‑research/rasr‑pairuav.
Authors:Peter Hollows
Abstract:
The multiplicative repetition penalty shipped across the LLM inference ecosystem (HuggingFace, vLLM, llama.cpp, and a dozen further engines) branches on the sign of each raw logit (divide positives by theta, multiply negatives). But the softmax is unchanged by adding a constant to every logit, so a model's logit zero‑point is arbitrary, and the sign‑branch reads that arbitrary point. The sign‑branch is itself the accepted fix for an earlier bug, so the accepted fix branches on a quantity the training objective leaves unconstrained. Two measurable consequences follow. (1) The penalty is not well‑defined: re‑centring a model's logits by a constant is a provable no‑op at theta=1, yet at a routine theta=1.3 it changes 58‑96% of greedy tokens, where subtractive and normalized penalties change none; real checkpoints sit at widely different zero‑points, so a fixed repetition_penalty is a different operation on every model. (2) It corrupts structured output: on 200 real‑world JSON schemas, theta=1.3 drops the rate of valid, schema‑conformant output from 97% to 23%. In our measurements, applying the penalty to normalized log‑probabilities instead of raw logits removes both effects. HuggingFace already ships that operator (LogitNormalization); today it is off by default and applied after the penalty. This note gives the mechanism, the measurements (five models up to 7B, base and RLHF, on WikiText‑103 prefixes; two code models on HumanEval and JSONSchemaBench; both effects replicated inside vLLM and llama.cpp through their own samplers on the same inputs), and the normalized variant.
Authors:Ni Yao, Zhenxu Wang, Danyang Sun, Chuang Han, Yanting Li, Jiaofen Nan, Fubao Zhu, Chen Zhao, Weihua Zhou
Abstract:
Alzheimer's disease (AD) is a common neurodegenerative disorder, and early diagnosis is of great significance for delaying disease progression and enabling timely intervention. Mild cognitive impairment (MCI), which represents an intermediate clinical stage between cognitively normal aging and AD. Structural magnetic resonance imaging (sMRI) provides detailed characterization of anatomical structures and plays an important role in AD‑related brain analysis. However, existing sMRI‑based brain network methods typically rely on a single graph construction strategy, limiting their ability to jointly capture spatial relationships and morphological similarities between brain regions. To address these issues, this paper proposes an sMRI‑based multi‑view masked graph neural network model (MVMGNN) for AD diagnosis. A joint node‑edge masking mechanism is proposed to simultaneously select radiomics feature dimensions and structural connections, reducing redundancy during graph learning. Furthermore, a patient‑level cross‑view gated fusion mechanism is proposed to integrate multi‑view representations. Experimental results on the ADNI dataset demonstrate that MVMGNN outperforms several competing approaches in AD classification. Interpretability analysis further demonstrates that MVMGNN is able to identify key brain regions associated with AD, providing useful insights into discriminative patterns in sMRI‑based brain networks.Our implementation is publicly available at https://github.com/chenzhao2023/MVMGNN_AD
Authors:Igor Itkin
Abstract:
A cheap swarm of unreliable agents can be steered to a correct consensus by a few strong, expensive "oracle" correctors. We ask how much one must spend, and where to place the oracles. We model the swarm as a consensus on a graph in which each oracle pins one node toward the truth at a cost‑coupled, concave strength, and measure quality by the coherence H(R)=tr M(R)^‑1. Our first result is that H stays submodular (each added oracle helps less than the last) even when the oracles differ in strength, so a cost‑benefit greedy comes within 1‑1/e of the best placement at any budget. Inverting the budget gives the budget‑correctness frontier B(eps), the least spend that guarantees an eps‑correct consensus: closed‑form on the complete graph, and a minimal oracle count k when oracles cost the same. Whether a budget then buys a few strong oracles or many medium onese curvature of the cost‑quality law: diminishing returns favour spreadsharply increasion. Measured onthe Qwen3 ladder (0.6‑32B), the law is concave for math verificatio convex foremergent code tracing, so the verdict is genuinely task‑dependent.https://github.com/YehudaItkin/budgeted‑oracle‑placemen
Authors:Tianwen Zhu, Hao Wang, Yonggang Wen
Abstract:
Public battery aging datasets are a critical asset for advanced health management, but their practical use is often limited by inconsistent formats, unclear schemas, and metadata scattered across repositories and publications. Current curation remains largely manual and hard to reproduce, while general‑purpose data integration tools miss the domain‑specific semantics of electrochemical time‑series data. We present BatteryLake, a governed data lakehouse that turns raw public battery data into benchmark‑ready assets through an agentic, physics‑grounded curation framework, with three contributions. First, LLM agents extract metadata and synthesize dataset‑specific converters, grounding every output in verbatim evidence and abstaining when none supports a value. Second, a human‑in‑the‑loop mechanism frames verification as selective prediction and gates admitted data through 26 schema, statistical, and physical‑plausibility rules. Third, we release an open benchmark of 41 datasets from over 25 institutions, with standardized SOH and RUL tasks, three split protocols, and eight baseline model families. The platform, benchmark, and curation protocol are publicly available at https://tianwen1209.github.io/batterylake/.
Authors:Mohammad Hosseini, Eray Erturk, Saba Hashemi, Maryam M. Shanechi
Abstract:
Large‑scale, multi‑subject widefield calcium imaging provides unprecedented access to brain‑wide cortical dynamics. However, the high dimensionality, complex spatiotemporal structure, and substantial task‑irrelevant activity in widefield recordings have largely restricted modeling efforts to single‑session analyses, limiting scalability and generalization. While multi‑subject pretrained models have been explored for some neural modalities, multi‑subject models for widefield calcium imaging have not yet been demonstrated; further, subject‑invariant zero‑shot behavior decoding remains elusive for multi‑subject models across neural modalities more broadly. As a first step toward foundation modeling of widefield data, we introduce WiCAT, a multi‑subject model that leverages self‑supervised pretraining to both outperform single‑session models and enable zero‑shot behavior decoding on unseen subjects. WiCAT introduces an atlas‑grounded tokenization scheme without session‑specific components and learns globally shared spatiotemporal representations. Across multiple widefield datasets, the pretrained model supports lightweight downstream decoding, transfers across subjects, tasks, and datasets, and outperforms baseline models. Notably, the model also achieves robust zero‑shot continuous behavior decoding and left‑out brain region reconstruction on unseen subjects.
Authors:Hamid Ebrahimy, Moritz Lucas, Martin Atzmueller
Abstract:
Machine Learning (ML) algorithms have been widely used to estimate agricultural variables across diverse contexts. However, because the quantity and quality of training data strongly influence performance of ML algorithms, their use can be constrained by limited or incomplete reference data. Synthetic Data Generation (SDG) offers a practical approach to address this issue by producing artificial but realistic samples that preserve key characteristics of the original data. Building on teacher‑student knowledge transfer and in‑context learning for tabular data, this study proposes a Task‑Conditioned SDG (TCSDG) algorithm that pairs a Bayesian Network generator with a transformer‑based tabular foundation model (TabICL). The proposed algorithm was evaluated on two agricultural prediction tasks: crop yield prediction and crop type classification. Six benchmark SDG algorithms were also utilized to compare their performance with that of TCSDG. Across twelve study sites, two training‑data fractions, four multiplication ratios, and three predictive ML algorithms, augmenting the original data with TCSDG‑generated synthetic data improved ML performance in 89% of the crop type classification experiments and 74% of the crop yield prediction experiments. TCSDG also substantially outperformed benchmark SDG algorithms and was the only method to consistently improve ML performance across both tasks at the aggregate level. The study demonstrates that carefully designed and processed synthetic data can improve ML performance in precision‑agriculture applications. TCSDG offers a practical and extensible framework for generating synthetic data that supports downstream ML agricultural prediction. The full implementation of TCSDG is publicly available as open source at https://github.com/HamidEbrahimy/TCSDG.
Authors:Jiayi Chen, Weiting Ou, Guangxu Zhu
Abstract:
WiFi sensing based on Channel State Information (CSI) promises ubiquitous, device‑free perception, yet current research remains trapped in a Tower of Babel ‑ fragmented into isolated silos where models are tailored to specific hardware dialects, fixed environments, and narrow tasks. The primary bottleneck is the Heterogeneity Gap: the disparity in signal dimensions, sampling rates, and semantic labels that prevents cross‑system understanding. To bridge this gap, we propose a foundation‑model framework that treats CSI not merely as raw signals but as a structured language with a learnable universal grammar. We first curate and standardize a large collection of heterogeneous real‑world CSI datasets, establishing a unified infrastructure that allows incompatible signal formats to be treated as a single corpus. Second, we introduce a modular architecture that acts as a universal translator where lightweight dataset‑specific adapters tokenize diverse signal inputs into a shared latent vocabulary, while a shared self‑supervised Transformer backbone learns the temporal syntax of human motion and environmental dynamics. This design decouples sensing semantics from hardware syntax. Extensive evaluations show that by mastering this universal language, our approach consistently outperforms task‑specific baselines and exhibits strong generalization capability in new environments, achieving superior efficiency in few‑shot scenarios. By effectively absorbing heterogeneity, the framework offers a path toward robust, general‑purpose wireless sensing, mirroring the linguistic generalization observed in Large Language Models. The code implementation is available at: https://github.com/cjychenjiayi/WiLLM.
Authors:Yifan Zhong, Zhang Chen, Tianrui Guan, Fanlian Zeng, Yuyao Ye, Tianjia He, Ka Nam Lui, Jiayi Li, Tingrui Zhang, Ruilin Yan, Xinhao Ji, Guangyu Zhao, Wenjie Lou, Jiayuan Zhang, Yuanpei Chen, Yaodong Yang
Abstract:
Steerability is a defining capability of generalist robot policies, yet remains largely absent in dexterous‑hand systems for lack of large‑scale, language‑aligned, and action‑accurate demonstration data. To address this bottleneck, we present a full‑stack system that scales dexterous VLA pre‑training from egocentric human videos and enables data‑efficient real‑robot post‑training. It integrates EgoSmith, a data pipeline that curates in‑the‑wild egocentric videos into 9.6K hours of high‑quality pre‑training data with 9x higher throughput and better accuracy than prior SOTA; a unified robot stack for teleoperation and human‑in‑the‑loop correction; and EgoSteer, a world‑model‑enhanced VLA trained on optimized infrastructure. Human‑data pre‑training equips EgoSteer with language‑guided manipulation priors, which are grounded through robot post‑training and improved by DAgger refinement. Empirically, EgoSteer robustly executes free‑form instructions across 40+ diverse tasks, demonstrating failure recovery, dexterity, and generalization. The pre‑trained model also few‑shot adapts to complex long‑horizon tasks, including box folding, on two embodiments with 75+% success. We open‑source the system, data, and model at https://egosteer.github.io/.
Authors:Jiayi Li, Kun Zhan
Abstract:
Existing safety mechanisms for multimodal large language models (MLLMs) face a fundamental trade‑off between safety and utility. Model fine‑tuning achieves robust safety but compromises general utility. Input‑side safety guardrails offer a lightweight alternative, yet they suffer from severe over‑refusal, indiscriminately blocking benign queries or those the model could have safely answered through refusal or advisory responses. We identify that the root cause of over‑refusal lies in the input‑aware paradigm: safety guardrails make safety decisions without considering whether the model itself is capable of generating safe responses. Usually, MLLMs already possess intrinsic safety mechanisms that can transform harmful inputs into harmless outputs, but input‑side safety guardrails override this capability, degrading user experience. Motivated by this insight, we propose a paradigm shift toward output‑aware safety guardrails. Our method operates within the model's hidden state space to predict whether the forthcoming generation will be unsafe before it is fully produced. By training a lightweight classifier via multi‑instance contrastive learning on hidden state representations, our approach distinguishes between inputs that will lead to unsafe outputs and those that will not, even when the inputs themselves contain risky elements. This enables precise intervention only when the model's actual response would be harmful. Extensive experiments demonstrate that our output‑aware safety guardrail matches the safety performance of existing methods while drastically reducing over‑refusal, preserving the model's utility and built‑in safety capabilities. Code is available at: https://github.com/kunzhan/OutGuard
Authors:Chengcheng Sun, Jiayun Tian, Cheng Zhai, Zhixiao Wang, Yajie Song, Xiaobin Rui, Jian Zhang, Philip S. Yu
Abstract:
Graph Neural Networks (GNNs) have emerged as a powerful paradigm in Knowledge Graphs (KGs) due to their intrinsic ability to model graph‑structured data. However, there remains a lack of a systematic review about GNN‑based methodologies across the entire knowledge graph technologies pipeline. To address this gap, we first propose a novel two‑level taxonomy framework for GNN‑based knowledge graph technologies: the KG technologies pipeline and GNN‑based perspective. Specifically, the knowledge graph technologies pipeline covers knowledge graph construction, knowledge graph embedding, knowledge reasoning and knowledge graph applications. Meanwhile, the GNN‑based perspective provides a new categorization of knowledge graph technologies with GNN models, such as GCN, GAT, and HGNN. Then, we analyze the advantages of GNN technology based on the characteristics of different tasks in the knowledge graph lifecycle. Furthermore, we detailed review various GNN‑based models for knowledge graph following the proposed taxonomy, and summarize strengths and limitations. Finally, we discuss unresolved challenges and outline promising directions for future research.
Authors:Haoyuan Li, Dizhe Zhang, Yuemei Zhou, Xiangkai Zhang, Haoran Feng, Xiaofan Lin, Wenjie Jiang, Bo Du, Ming-Hsuan Yang, Lu Qi
Abstract:
In this work, we aim to address the challenge of long‑range memory in panoramic world models by exploiting the rotation‑equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current‑action modeling and long‑range memory through Dense Panoramic Ray‑Conditioning (DPRC) and Geometry‑aware Memory Augmentation (GMA).Then, a three‑stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large‑scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large‑scale dataset consisting of both real‑world video clips collected via panoramic unmanned aerial vehicles and high‑quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and dataset will be publicly available. More information can be found on our project page: https://lihaoy‑ux.github.io/panoworld‑page/.
Authors:Mingyang Huang, Peng Zhang, Li Hu, Guangyuan Wang, Ruoshi Zhang, Yi Lu, Gang Cheng, Bang Zhang
Abstract:
Generating long‑duration, high‑definition, and rhythmically synchronized dance videos directly from music remains a significant challenge, primarily due to the temporal constraints of current diffusion models, which typically fail beyond 20 seconds. Existing approaches, whether they rely on intermediate 3D skeletons or on end‑to‑end video synthesis, suffer from temporal drift, identity inconsistency, and repetitive motion patterns when extended to longer horizons. To address these limitations, we propose a novel hierarchical framework for minute‑scale coherent music‑to‑dance generation. Our method decouples the process into global keyframe planning and local temporal refinement, leveraging full‑track musical context to ensure long‑range coherence. Key innovations include dynamic frame rate adaptation via time‑mapped RoPE embeddings for precise alignment, an optical‑flow‑based loss function to enhance motion continuity, and motion‑speed control to preserve high‑fidelity details during rapid movements. Extensive experiments demonstrate that our framework surpasses the conventional duration barrier, generating stable, 720p/30fps videos exceeding one minute with superior temporal stability. Furthermore, the model exhibits robust versatility across five distinct dance genres, conditioned on both audio and textual prompts, establishing a new state‑of‑the‑art in coherent, long‑form dance video synthesis.
Authors:Jiawen Li, Tian Guan, Huijuan Shi, Xitong Ling, Mingxi Fu, Anjia Han, Chao He, Yonghong He
Abstract:
Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi‑stage agglomerative distillation that sequentially distills eight vision‑only, vision‑language, and slide‑level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile‑level pathology images and 155,604 high‑resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region‑of‑interest tissue analysis, vision‑language multimodal evaluation, and whole‑slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task‑matched pathology foundation models. These results demonstrate that agglomerative distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at https://github.com/WonderLandxD/ALICE.
Authors:Sithu Aung, Viktor Kocur, Yaqing Ding, Torsten Sattler, Zuzana Kukelova
Abstract:
Global Structure‑from‑Motion (SfM) is an efficient paradigm for recovering camera poses and sparse 3D structure from unordered images. However, its reliance on scale‑ambiguous epipolar geometry makes global positioning sensitive to noisy baseline estimates and weak view‑graph constraints, while false edges from visually ambiguous pairs can further degrade reconstruction. We propose DGSfM, a depth‑aware global SfM pipeline that uses monocular depth maps as a scalable prior while preserving explicit multi‑view optimization. For each image pair, we use a depth‑aware relative pose solver to convert scale‑ambiguous epipolar constraints into scale‑aware relative pose constraints. We further improve robustness through view‑graph filtering and depth‑consistency‑based correspondence pruning, which suppress false edges and matches that remain plausible under epipolar geometry alone. Finally, global scale averaging and depth‑guided pose‑point initialization align monocular depth maps into a common reconstruction scale and provide stable initialization for global positioning and bundle adjustment. Experiments on ETH3D and IMC2021 show that DGSfM consistently improves over strong global SfM baselines across sparse and dense matching front‑ends, achieving substantial gains in pose accuracy. Code is available at https://github.com/sithu31296/DGSfM.
Authors:Lihe Yang, Zhen Zhao, Hengshuang Zhao
Abstract:
High‑quality visual representation is a long‑standing pursuit in computer vision. In the context of multimodal LLMs (MLLMs), feeding higher‑resolution images can produce more fine‑grained visual tokens. However, it introduces additional computational and design complexity, due to multiple forward passes and post‑processing of increased tokens. Before simply adopting a higher resolution, have we truly unlocked the model's full perception capability at a standard resolution? Therefore, we study an interesting problem: how to achieve fine visual perception under lower cost without larger images. We present SigLIP‑HD in this work. The core is a highly simple fine‑to‑coarse supervision design. We enforce the coarse feature of a mid‑resolution image to mimic the fine‑grained feature of its high‑resolution version. We build this framework on the advanced SigLIP 2 model. Our final model produces better visual tokens at exactly the same inference budget. It is validated on extensive MLLM benchmarks and consistently delivers stronger results than our baseline model, especially on OCR‑related tasks.
Authors:Johannes Schmitt, Tim Gehrunger, Jasper Dekoninck, Gergely Bérczi, Uri Kreitner, Liam Price, David Holmes
Abstract:
Large language models (LLMs) have shown increasing promise in solving open problems in mathematics. However, their performance can be further improved through agentic workflows tailored to real‑world mathematical practice. To this end, we introduce ProofCouncil, a mathematical agent that is designed to tackle open problems using an author‑critic architecture. ProofCouncil served as a submission to the second batch of FirstProof, a challenge consisting of 10 real‑world mathematical problems that agents must solve autonomously. Its submissions for 6 of the 10 problems were judged by the referees to be correct up to at most minor revisions, showing the best performance among participating teams. We also evaluate ProofCouncil on 30 open problems collected from mathematical researchers. Among the 21 solutions that received human feedback, 5 were judged completely correct, 2 more were judged promising pending final verification, and a further 8 contained useful partial progress. In this short paper, we describe the development of ProofCouncil and the agent‑building library used to create it, which we release as open source to the community.
Authors:Hyungtae Lim, Nathan Hughes, Xihang Yu, Ruihan Xu, Yun Chang, Jingnan Shi, Rajat Talak, Luca Carlone
Abstract:
3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph systems model object geometry coarsely, relying on partial point clouds or class‑level CAD templates, which limits instance‑specific shape detail. This paper presents Hydra++, a system‑level investigation into how learning‑based object shape estimators can be integrated into a hierarchical 3D scene graph pipeline. Hydra++ incorporates category‑agnostic shape estimation and a reprojection‑mask consistency check to reject degenerate predictions from partial observations or imprecise segmentation. In its default CRISP‑based configuration, Hydra++ performs online scene graph construction; slower estimators such as SAM3D are evaluated as modular alternatives to demonstrate generalization‑latency trade‑offs. Furthermore, to address the challenges of sparse and noisy depth measurements in outdoor environments, Hydra++ supports a hybrid LiDAR‑camera configuration for large‑scale operation, improving scene‑level reconstruction quality. Experiments in both simulation and real‑world outdoor campus scenarios demonstrate that Hydra++ improves object‑ and scene‑level reconstruction quality. Project page is available at https://hydra‑plusplus.github.io/.
Authors:Anil Osman Tur, Tonje Knutsen Sordalen, Kim Tallaksen Halvorsen, Cigdem Beyan
Abstract:
Long‑term animal re‑identification (ReID) must remain robust to gradual morphological evolution and seasonal appearance shifts. Although recent vision‑language models provide strong pretrained visual representations, adapting them to longitudinal ecological settings remains challenging, particularly under identity and temporal distribution shifts. We present a parameter‑efficient CLIP adaptation framework for animal ReID and introduce a continuous metadata‑conditioning mechanism that incorporates numerical attributes directly into the prompt representation during training. While low‑rank visual adaptation, prompt‑based supervision, and cross‑modal alignment provide the adaptation framework, the proposed metadata‑conditioning strategy constitutes the primary methodological contribution. By preserving the continuous structure of numerical metadata rather than discretizing it into textual categories, the proposed approach enables smooth modulation of the embedding space during training while maintaining a purely visual inference pipeline. Experiments on a seven‑year longitudinal fish dataset and multiple wildlife benchmarks demonstrate improved performance under closed‑set, open‑set, and time‑aware evaluation protocols. The results demonstrate that continuous metadata conditioning improves robustness to longitudinal appearance variation and temporal distribution shifts, while parameter‑efficient adaptation enables a purely visual inference pipeline without requiring metadata at test time. Code and evaluation splits can be found at: https://github.com/AnilOsmanTur/MetaPrompt‑ReID.
Authors:Hyein Park, Namho Kim, Junhwa Kim
Abstract:
Ambivalence and hesitancy are subtle behavioral states that are expressed through a combination of verbal content, facial behavior, visual context, and acoustic cues. Effective recognition therefore requires not only extracting informative unimodal representations, but also modeling how temporally aligned behavioral evidence interacts across modalities. In this paper, we propose a synchronized visual‑facial cross‑refinement framework (SVF‑CR) with pairwise multimodal evidence fusion for ambivalence and hesitancy recognition. The proposed method first extracts whole‑video segment tokens and cropped‑face segment tokens using the same temporal partition. The synchronized visual and facial tokens are refined through intra‑modal self‑attention and bidirectional visual‑facial cross‑attention, allowing whole‑video context and local facial behavior to mutually refine each other before evidence construction. We then construct segment‑level visual‑facial evidence using consistency and discrepancy modeling, followed by temporal self‑attention and attention pooling. Textual and acoustic features are lightly refined through context self‑attention and are fused with the enhanced visual‑facial evidence at the final decision stage using pairwise evidence fusion. Experiments on the BAH (Behavioral Ambivalence/Hesitancy) public evaluation split show that the proposed synchronized visual‑facial cross‑refinement improves public macro‑F1 over both global visual‑face token fusion and synchronized evidence baselines, achieving a public macro‑F1 of 0.7156. Code is available at : https://github.com/hiinnnii/BAH‑Challenge‑ECCV2026\_SVF‑CR.
Authors:Raza Yunus, Benjamin Ummenhofer, Jan Eric Lenssen, Eddy Ilg
Abstract:
Decomposing outgoing surface radiance into material and illumination during inverse rendering is essential for applications such as relighting and augmented reality, yet it is severely ill‑posed since multiple combinations can result in the same observed colour. Capturing an object under multiple lighting conditions usually helps resolve this ambiguity as it constrains the optimization towards correct solutions. In this work, we explore the potential of reconstructing rigidly moving objects ‑‑ which provides observations of diverse light‑surface interactions ‑‑ to resolve the material‑lighting ambiguity in inverse rendering. For this purpose, we introduce a relightable approach that marries object tracking and reconstruction with inverse rendering for general rigidly moving objects. Our experimental analysis on synthetic data demonstrates that motion can be an advantage for disentangling material and lighting: the reconstructed material is significantly more accurate when the object is observed under rigid motion than when it is static. Moreover, results on RGB videos of real hand‑held objects show that our pipeline preserves this advantage even under noisy real‑world conditions.
Authors:Ivan Ilin, Philip Zmushko, Peter Richtárik
Abstract:
Large language models (LLMs) remain expensive to fine‑tune because full‑parameter updates require substantial memory, compute, and per‑task storage. We study whether saliency signals originally developed for pruning can be reused to choose where a model should adapt. We propose Super, a sparse parameter‑efficient fine‑tuning (PEFT) method that fixes a small trainable support using a Wanda‑style activation‑weighted magnitude score [Sun et al., 2023] computed from a calibration pass. We then introduce Supra, a hybrid adapter that combines this sparse update with LoRA while preserving a matched trainable‑parameter budget through a simple budget‑splitting rule. In single‑seed Math17K arithmetic experiments on Llama‑3.2‑1B and Meta‑Llama‑3‑8B, the best Super/Supra variants achieve the highest average accuracy among the tested schedule‑selected adapter configurations. We also include a PaFi‑style magnitude‑only support as a closest training‑free sparse baseline and find that low‑score supports under both magnitude and Wanda‑style orderings can be effective. These results suggest that simple pruning‑inspired orderings can provide useful fixed sparse supports for PEFT, especially when combined with low‑rank adapters.
Authors:Libo Lin, Shuangli Du, Minghua Zhao, Zhenzhen You, Shun Lv, Yiguang Liu
Abstract:
Generally, monocular methods capture rich contextual priors but lack geometric precision, whereas stereo methods are geometrically accurate yet struggle in textureless and occluded regions. Several approaches attempt to combine their strengths to enhance the generalization of stereo matching (SM) by aligning monocular depth with stereo information. However, establishing a stable and generalizable alignment is challenging, and unreliable monocular cues can substantially degrade performance. This paper rethinks monocular depth embedding. First, to prevent shortcut learning, we reduce branch coupling instead of expanding network width. Second, we construct soft constraints instead of hard ones from monocular depth to improve tolerance to monocular depth errors. Based on the principles, we integrate monocular information into both feature extraction and GRU iterations. Specifically, the monocular depth map is fused with the RGB image to sharpen depth boundary perception and suppress matching ambiguities. The fused image is then used for feature extraction, allowing the contextual features to encode global geometric information. Furthermore, the monocular depth gradient feature is employed to guide disparity updates, helping to escape local oscillations. Finally, to address the boundary blurring of supervised disparity caused by data augmentation, we propose an edge confidence estimation method and an edge‑aware loss function. Our method achieves state‑of‑the‑art (SOTA) performance on multiple standard benchmarks, demonstrating excellent generalization while improving accuracy. The code is available at https://github.com/linliboabc‑maker/stereo‑matching‑digital.
Authors:Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra, Matthias Hein
Abstract:
Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety. Yet existing benchmarks measure it unreliably. They miss knowledge that resurfaces under paraphrased or indirect queries, a failure we call under‑forgetting, and lack the semantic, syntactic, and lexical probes needed to verify that unrelated knowledge is preserved, a failure we call over‑forgetting. Both failures reflect an asymmetric generalization problem. Forget evaluation must cover diverse query formulations of the same target facts, testing whether forgetting holds beyond exact training prompts. Retain evaluation must probe a far larger and implicitly defined set, namely every fact disjoint from the forget target. The retain set thus defines the effective forget set, yet current datasets provide no fine‑grained annotation of this forget‑retain boundary. We address this with SUITE, an evaluation protocol and training corpus that captures forget‑retain structure for real‑world factual domains. Methods trained on SUITE improve substantially, showing that training data is as important as algorithmic design. Building on the obtained insights, we introduce JensUn++, an unlearning algorithm that achieves the best forget‑retain utility trade‑off across three LLMs, in both sequential and joint unlearning settings. Code and datasets are available at https://amitpeleg.github.io/forget‑narrowly‑retain‑broadly
Authors:Junyuan Deng, Heng Li, Kejie Qiu, Lingteng Qiu, Rui Peng, Weichao Shen, Weihao Yuan, Siyu Zhu, Zilong Dong, Ping Tan
Abstract:
Recent 3D geometric foundation models, such as VGGT, provide robust feed‑forward 3D reconstruction by directly predicting camera poses and 3D scene points from input images. However, their results remain inaccurate, and scaling them to long sequences or large unordered image sets typically requires chunk‑wise processing, which can introduce drift and inconsistency. We present Glob3R, a global SfM‑style reconstruction built on 3D foundation models. Our key idea is to explicitly optimize feed‑forward geometric predictions. To this end, we augment a frozen Pi3X backbone with a lightweight dense matching head that predicts image warps between selected reference frames and neighboring views. These dense warps are converted into sparse but reliable multi‑view feature tracks, which provide correspondence constraints for global optimization. We further introduce a keyframe‑based sliding‑window association strategy that propagates tracks and relative poses across overlapping windows, enabling scalable reconstruction. Finally, we perform global motion averaging and bundle adjustment to refine camera poses, reduce scale inconsistencies, and recover dense scene geometry. Extensive experiments on indoor, outdoor, large‑scale driving, and unordered SfM benchmarks demonstrate that Glob3R achieves robust and accurate reconstruction. It consistently improves over feed‑forward foundation‑model baselines and recent scalable reconstruction methods, while being more robust than classical SfM pipelines. The refined poses also lead to higher‑quality neural rendering, validating the benefit of combining foundation‑model priors with global geometric optimization. Project page: https://junyuandeng.github.io/Glob3r
Authors:Matěj Kripner, Milan Straka
Abstract:
In this system paper, we present OpenProver, an open‑source system for LLM‑driven automated theorem proving (ATP) with integrated Lean 4 formal verification. OpenProver integrates a Planner‑Worker‑Verifier architecture inspired by recent ATP agentic systems such as Aletheia. A Planner agent maintains a compact Whiteboard scratchpad and an unbounded Repository of intermediate findings, and decomposes mathematical work into parallel Workers. OpenProver is fully open‑source, offers reproducible evaluation through automatic formal verification of generated proofs, and provides an interactive terminal interface for human‑guided proof search. In interactive mode, OpenProver allows the human operator to monitor and steer the proof search process, motivated by the established human‑AI synergy in interactive code generation. To showcase the potential for quantitative ablation experiments enabled by automatic formal verification, we evaluate OpenProver on ProofNet and compare it with a simple baseline. OpenProver is publicly available at https://github.com/kripner/OpenProver.
Authors:Seyyedhamid Azimidokht, Mehdi Monemi, Abdelhak Kharbouch, Farid Hamzehaghdam, Mehdi Rasti, Jamshid Aghaei, Emil Kurvinen
Abstract:
The rapid expansion of solar photovoltaic (PV) systems has increased the need for reliable and scalable fault classification, as manual inspection is impractical at scale. Thermal infrared (IR) imaging provides a non‑contact solution for identifying PV faults; however, accurate classification remains challenging due to class imbalance, limited texture information, and subtle thermal differences. In this work, we investigate the applicability of Joint‑Embedding Predictive Architecture (JEPA) for thermal IR PV fault classification across various scenarios and propose JEFFNet (JEPA‑EFFicientNet), a multibranch architecture that combines JEPA‑based self‑supervised representation learning with EfficientNetV2‑S‑based supervised convolutional feature extraction. JEFFNet fuses semantic representations from a JEPA‑pretrained Vision Transformer with convolutional features from EfficientNetV2‑S, enabling complementary feature learning. JEFFNet is evaluated on two public thermal IR datasets, PVF‑10 and InfraredSolarModules (ISM), for both multiclass and derived binary (healthy/faulty) classification. On PVF‑10, JEFFNet achieves an F1‑score of 93.21 and an accuracy of 94.33 in the 10‑class task, and an F1‑score of 97.53 and an accuracy of 96.41 in the derived 2‑class task. On ISM, JEFFNet achieves an F1‑score of 72.60 and an accuracy of 83.88 in the 12‑class task, and an F1‑score of 94.69 and an accuracy of 94.78 in the derived 2‑class task. JEFFNet also uses only 108.6M parameters versus 205.91M for GEPFNet, a 47.2% reduction. These results demonstrate that combining self‑supervised semantic and supervised convolutional features provides an effective, parameter‑efficient solution for thermal IR PV fault classification. The source code is publicly available at https://github.com/Azimi2kht/JEFFNet
Authors:Suting Ni, Hanbing Zhang, Zhenyu Wei, Guo Chen, Chixuan Zhang, Ye Shi, Jingya Wang
Abstract:
Tactile feedback is fundamental to Hand‑Object Interaction (HOI), governing contact formation, force regulation, and stable manipulation, making it essential for achieving true human‑like dexterous manipulation. Yet, current human‑to‑robot dexterous transfer pipelines primarily rely on kinematic trajectories, resulting in motion imitation without physically grounded interaction. To address this, we introduce TactiDex, a real‑world tactile‑guided benchmark specifically designed to move dexterous manipulation beyond kinematic mimicry toward contact‑level human‑likeness. TactiDex provides a comprehensive dataset that elegantly aligns whole‑hand tactile signals with multi‑granularity kinematic and object states, coupled with standardized evaluation metrics. Building upon this data paradigm, we propose a tactile‑driven transfer framework that effectively translates human demonstrations into physically plausible robotic execution. We introduce TactiSkill, a framework built upon a novel tri‑component tactile reward that innovatively uses tactile signals as structured supervision. This reward unifies guidance, human‑like alignment, and contact constraints into a single objective. Through comprehensive experiments on both single and bimanual tasks, we demonstrate that TactiSkill achieves superior performance in manipulation success and physical realism. This work lays a crucial foundation for advancing tactile‑aware dexterous manipulation. Our project page at https://tactidex.github.io/.
Authors:Qiwei Yang, Pingping Zhang
Abstract:
Aerial‑Ground Person Re‑IDentification (AG‑ReID) aims to retrieve the same person across heterogeneous aerial and ground camera platforms. Although great progress has been made, existing methods remain suboptimal due to the direct feature alignment across views, overlooking view‑specific cues. To address this issue, we propose a novel Hierarchical Hyperbolic Representation (HiHR) framework for AG‑ReID. More specifically, we first extract multi‑granularity features based on pre‑trained visual‑text encoders. Then, we propose a Text‑guided Multi‑granularity Fusion (TMF) to fuse multi‑granularity features and enhance the representation ability of identity features. Furthermore, we introduce the Hierarchical Hyperbolic Learning (HHL) to construct a hierarchical feature structure in a hyperbolic space. This hierarchy includes a coarse level that ensures identity separability and cross‑view consistency, and a fine level that preserves view‑specific discriminative cues. As a result, our proposed framework can effectively aggregate view‑invariant and view‑specific discriminative features for AG‑ReID. Extensive experiments on four AG‑ReID benchmarks demonstrate the effectiveness of our framework. The source code is available at https://github.com/YangQiWei3/HiHR.
Authors:Chenxu Peng, Chongtian zhou, Dicheng Liu, Bo-Wen Yin, Yimian Dai, Xialei Liu, Ming-Ming Cheng, Xiang Li
Abstract:
Fusing standard RGB frames with asynchronous event streams has emerged as a definitive paradigm for robust perception in degraded environments. Although unified backbones have recently gained traction in multi‑modal vision, adapting them to the RGB‑Event domain remains fundamentally challenging. Existing architectures either resort to decoupled dual encoders that double computational overhead, or adopt generic unified designs that fail to resolve implicit geometric parallax and cross‑spectral aliasing under the extreme representational divide between dense intensity grids and sparse kinematic spikes. To transcend these bottlenecks, we present Evita, the first unified backbone specifically engineered for dedicated dense RGB‑Event parsing. To achieve profound modal synergy, Evita explicitly embeds a suite of intrinsic co‑learning modules directly into every encoder layer. Specifically, it features Geometric Parallax Rectification for adaptive spatial alignment, Harmonic Spectral Resonance for texture transfer exclusively in the complex frequency domain, and Transient Global Routing for event‑driven asymmetric attention. To guarantee robust feature extraction against spatial misalignments and decouple representations from specific event encodings, we construct N‑ImageNetV2 alongside a stochastic event representation mixing pretraining protocol, empowering the network to seamlessly accommodate arbitrary event formats in downstream tasks. Extensive evaluations across the DELIVER, DDD17, and DSEC benchmarks confirm that Evita establishes new state‑of‑the‑art metrics while delivering a superior accuracy‑latency trade‑off for real‑time multimodal perception.The code are publicly available at: https://github.com/chaineypung/Evita.
Authors:Sang-Hoon Lee, Ha-Yeong Choi
Abstract:
Representation alignment (REPA) has been investigated to accelerate diffusion training, but we observe that regularizing intermediate representations in diffusion Transformers (DiT) may implicitly entangle latents and limit generative capacity. To address this issue, we propose ReGen, a hierarchical multi‑prompt representation generation framework that jointly estimates multiple vector fields for both representations and data within a single diffusion model. We further introduce generalized flow matching (GFM) to improve the generalization of conditional flow matching (CFM). We validate ReGen on single‑stage waveform diffusion models including neural audio codec and Wave‑VAE. ReGen significantly improves waveform generation quality from highly compressed latent representations at 12.5 Hz. We also present ReGenVoice, a latent diffusion model (LDM)‑based text‑to‑speech model that achieves strong speech intelligibility (WER) and speaker similarity (SIM) with a small dataset. Moreover, operating the LDM at 6.25 Hz with rich semantic and acoustic latent representation enables efficient training and sampling, requiring only 1 day of training on 4 GPUs and fast inference with an RTF of 0.08. Audio samples are available at https://regenvoice.github.io/demo/.
Authors:Junyi Hu, Zhewen He, Haomian Huang, Aoxiang Yang, Yi Fang
Abstract:
Sign language translation (SLT) converts continuous sign videos into spoken language text. Gloss‑free approaches leverage pre‑trained visual encoders and language models but rely on implicit cross‑modal alignment from translation supervision alone. We present VTaMo, a framework that introduces explicit multi‑granularity alignment at three levels: (1) local alignment via entropy‑regularized optimal transport with a learnable null token for fine‑grained frame‑to‑token correspondences; (2) global alignment via a learnable orthogonal transformation that calibrates embedding space geometry through Earth Mover's Distance; and (3) position‑aligned contrastive learning for discriminative token‑level representations. Experiments on Phoenix‑2014T, CSL‑Daily, How2Sign, and OpenASL demonstrate consistent state‑of‑the‑art performance, with ablations confirming the complementary contributions of each component. Code is available at https://github.com/junyi2005/vtamo.
Authors:Minhyuk Hwang, Sangmin Kim, Seunguk Do, Daneul Kim, Jaesik Park
Abstract:
Existing volumetric capture of dynamic human performance achieves high fidelity with dense camera arrays. However, in real‑world scenarios, only a handful of low‑overlap cameras are available, which degrades the output quality and leaves large areas unobserved. Recent 4D reconstruction methods have focused on low‑overlap settings, yet they still produce noticeable artifacts in under‑observed regions. Video diffusion models have emerged as another option, but they show geometrically inconsistent results for humans. To address these limitations, we propose StudioRecon, a pipeline that reconstructs 4D human scenes from sparse, low‑overlap cameras by decoupling background and humans. We densify background supervision by synthesizing hundreds of camera‑controlled novel views with a video diffusion model. We also robustly initialize deformable Gaussian humans with cross‑view identity association and triangulated multi‑view keypoint fitting. Finally, our recursive enhancement module with motion‑adaptive consistency injection harmonizes the composed output, thereby further avoiding remaining artifacts. We achieve state‑of‑the‑art novel view synthesis across four real‑world datasets and demonstrate applications such as novel trajectory rendering and human replacement.
Authors:Guohuan Xie, Mengqi Lei, Chuan Shi, Wei Bao, Yue Gao, Siqi Li
Abstract:
Although Video Large Language Models (Video LLMs) have shown strong performance in video understanding, their efficiency is still limited by the large number of visual tokens. Existing video token compression methods typically rely on frame‑wise saliency or heuristic token merging, which can over‑focus on locally salient regions and produce ambiguous fused features. To address these issues, we propose GeoTrace, a training‑free spatiotemporal token compression framework that decomposes video evidence into exact skeleton tokens and traceable residual event tokens. Specifically, Contextual Farthest‑Point Anchoring (CFPA) preserves salient, context‑consistent, and high‑coverage skeleton tokens, while Trajectory‑Constrained Residual Condensation (TCRC) compresses residual tokens through one‑to‑one temporal trajectories and constrained near‑manifold condensation, producing traceable event tokens with reduced ambiguity. We evaluate GeoTrace on four Video LLMs across four video understanding benchmarks, and the results demonstrate its effectiveness and generalization across different model architectures and scenarios. On LLaVA‑OneVision, with only 10% visual tokens retained, GeoTrace achieves a \(12.99×\) TFLOPs reduction while preserving 99.1% of the vanilla performance. Overall, GeoTrace offers a compact and traceable token representation for efficient and robust Video LLM inference. Code is available at \hrefhttps://github.com/guohuan‑xie/GeoTrace.git\textttCode.
Authors:Tianpeng Liu, Xinhua Jiang, Li Liu, Qinmu Shen, Siwei Tang, Zhen Liu, Yongxiang Liu
Abstract:
Object detection is a fundamental component in numerous Unmanned Aerial Vehicle (UAV) applications, yet it has long been plagued by hindrances like occlusion or target pixel scarcity. Active Object Detection (AOD) provides a novel paradigm to address these challenges via active vision, while UAV‑based AOD research remains scarce due to the lack of high‑quality datasets and benchmarks for algorithm development and evaluation. To fill this gap, this paper presents ATRNet‑LUDO, the first large‑scale real‑world dataset for UAV‑Ground Active Object Detection (UGAOD). It contains 121,000 multi‑view panoramic multi‑target aerial images and 1.21 million local single‑target slices, covering 10 vehicle targets across 40 scenarios. It enables the construction of diverse training and testing environments for UAV agent interaction and active observation policy learning. Based on this dataset, we establish a comprehensive evaluation benchmark for AOD policy learning methods. Most existing AOD policies rely on Deep Reinforcement Learning (DRL) but suffer from poor generalization. Evaluations on our benchmark reveal a significant generalization gap between training and testing performance, highlighting an urgent need for solutions. To this end, we leverage the Joint Embedding Predictive Architecture (JEPA) to construct a world model that enhances state representation learning, and propose AOD‑JEPA by incorporating AOD‑specific prior knowledge. Extensive experiments validate its effectiveness and superiority. We hope ATRNet‑LUDO and the benchmark will advance research in the UGAOD field. The dataset and code are soon available at https://github.com/Leo000ooo/LUDO_dataset.
Authors:Yang Chen, Yunwen Li, Yufan Shen, Minghao Liu, Tianyu Zheng, Bin Fu, Qunshu Lin, Zhi Yu, Botian Shi
Abstract:
Recent advancements in LVLMs necessitate robust benchmarks for complex, visually grounded reasoning. A critical limitation is identified in many document understanding benchmarks: visual content is often reducible to text, enabling high performance without genuine visual grounding. To address this limitation, OmniMapBench is introduced to foster visual‑centric reasoning for map documents. The benchmark comprises 2,096 manually annotated question‑answer pairs across 1,603 map documents from nine categories. It is designed to probe a hierarchy of skills, ranging from perception to multi‑step visual reasoning. To quantify benchmark properties, a simple yet effective benchmark‑level metric is proposed: the Visual Dependency Index (VDI), defined as the accuracy drop when images are replaced with question‑agnostic descriptions. OmniMapBench exhibits higher VDI than established benchmarks, which quantitatively validates its focus on irreducible visual reasoning. Comprehensive evaluations of 25 leading LVLMs are conducted on OmniMapBench. A significant performance gap is observed, with the top‑performing model achieving only 75.03% accuracy. This result underscores the challenges posed by OmniMapBench to current LVLMs. This work aims to catalyze progress in visual‑centric reasoning for document understanding of LVLMs. The dataset and code are publicly available at https://github.com/SIGMME/OmniMapBench.
Authors:Shuo Huai, Hao Kong, Shiqing Li, Xiangzhong Luo, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu
Abstract:
Edge devices are increasingly utilized for deploying deep learning applications on embedded systems. The real‑time nature of many applications and the limited resources of edge devices necessitate latency‑targeted neural network compression. However, measuring latency on real devices is challenging and expensive. Therefore, this letter presents a novel and efficient framework, named EvoLP, to accurately predict the inference latency of models on edge devices. This predictor can evolve to achieve higher latency prediction precision during the network compression process. Experimental results demonstrate that EvoLP outperforms previous state‑of‑the‑art approaches by being evaluated on three edge devices and four model variants. Moreover, when incorporated into a model compression framework, it effectively guides the compression process for higher model accuracy while satisfying strict latency constraints. We open source EvoLP at https://github.com/ntuliuteam/EvoLP.
Authors:Shaoxiang Wang, Kejia Zhang, Haiwei Pan, Lan Zhang
Abstract:
Cross‑view geo‑localization (CVGL) aims to achieve GPS‑free localization by matching drone‑view images with corresponding satellite‑view images. Existing supervised methods rely on large‑scale manually annotated cross‑view image pairs, making them costly and difficult to scale. In contrast, existing unsupervised approaches typically depend on generative models or clustering‑based stage‑wise optimization, which are prone to distribution bias and the accumulation of noisy pseudo‑labels. To address these limitations, we propose STEAM (Stable Self‑Training with Elastic Matching and Adaptive Purification), an end‑to‑end unsupervised cross‑view geo‑localization framework that performs self‑training directly on real drone and satellite images. Specifically, the proposed Stable Spatial‑Aware Module enhances the stability of feature representations, Elastic Matching discovers high‑quality cross‑view pseudo‑labels, and Adaptive Purification dynamically maintains a reliable pseudo‑label repository throughout the self‑training process. Extensive experiments on the University‑1652 and SUES‑200 benchmarks demonstrate that STEAM achieves state‑of‑the‑art performance among all existing unsupervised methods and delivers performance comparable to supervised approaches, validating the effectiveness and superiority of the proposed framework. The source code is available at https://github.com/wsx‑heu/STEAM.git.
Authors:Shrimon Mukherjee, Kishalay Das, Partha Basuchowdhuri, Pawan Goyal, Niloy Ganguly
Abstract:
Graph Neural Networks have emerged as a powerful tool for the fast and accurate prediction of various crystal properties. These models often encode domain‑specific knowledge into their graph encoding modules, which increases their parameter size and makes their performance heavily dependent on domain expertise. Added to this, explicitly incorporating all chemical and structural features, that might influence a specific crystal property into the GNN encoder, is a challenging task. In this work, we propose a soft prompt learning framework that captures latent features essential for property prediction, which are not explicitly provided to the GNN. We introduce a novel multilevel graph prompt learning framework comprising both node‑level and graph‑level soft prompts. At the node level, we capture the local chemical semantics of different atom types, while at the graph level, we encode the global structural symmetry of the crystal graph. Our proposed prompt learning framework is lightweight and seamlessly integrates with any existing GNN encoder. Extensive experiments on popular benchmark datasets show that incorporating prompt learning significantly improves (3% ‑ 15%) the performance of state‑of‑the‑art GNN models in crystal property prediction tasks. Furthermore, the learned soft prompts enable cross‑property knowledge transfer, enhancing prediction performance for properties with limited training data. Code is available at https://github.com/shrimonmuke0202/Prompt.git
Authors:Joseph K. Miller
Abstract:
We formalize a research result in the Lean 4 proof assistant by having a mathematician direct an AI system, and frame the activity as a formalization game. The objective is to turn a LaTeX document into Lean. The game is won when the development compiles, contains no sorry, and a machine check shows the target theorems rest on Lean's foundational axioms alone. Reuse is a second check, by a definition we introduce: whether the development yields a self‑contained layer of general mathematics the wider library could absorb. The case study is a complete, axiom‑clean formalization of well‑posedness for the nonlinear Vlasov equation via Dobrushin's mean‑field route ‑‑ existence, uniqueness, the stability estimate and mean‑field limit, and a short‑window superposition principle (weak solutions are Lagrangian). The human's role was to direct, not to write proofs: to scope the definitions, steer the decompositions, and triage the library's gaps; the AI agent executed. The formalization certifies the proof of each statement as written; whether the written statement is the intended theorem stays the mathematician's judgment. The optimal‑transport machinery that fell out of the build (in particular, properties of the Wasserstein‑1 metric and the Kantorovich‑Rubinstein duality theorem) separates into a self‑contained layer that compiles against Mathlib alone: about a sixth of the development (49 of 299 declarations), behind a 22‑declaration interface with no reverse dependency. The headline theorems ran in about a week, the full development in about a month. We report the quantitative claims as observations of one game, not as general laws. The game's rules name no particular system, so the methodological framing is meant to outlast the tools of any one run.
Authors:Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya, Kuniaki Saito, Cristian C. Beltran-Hernandez, Mai Nishimura
Abstract:
Vision‑language‑action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large‑scale post‑training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control. Directly converting pretrained VLMs into VLAs with minimal architectural change offers a more transparent path to understanding how VLM capabilities transfer across model scales. The core obstacle is output‑distribution mismatch: predicting actions as bare numeric token sequences moves generation away from the VLM's pretrained language distribution, degrading the capabilities we seek to preserve. To address this, we propose CLAP (Causal Language‑Action Prediction), which prepends each numeric action sequence with a natural‑language action description, causally conditioning precise action‑token prediction on a language‑action plan without modifying the backbone architecture. With single‑epoch fine‑tuning alone, 2B CLAP achieves 90.8% on LIBERO (+14.9 pt over VLA‑0) and improves robustness on LIBERO‑PRO under language, object, and spatial perturbations. We will release CLAP at 0.8B, 2B, and 4B as an open‑weight, multi‑scale compact VLA family from a single VLM lineage, enabling controlled analysis of VLM‑to‑VLA capability transfer.
Authors:Hyung-Jin Yoon, Hunmin Kim
Abstract:
We study Model Predictive Path Integral (MPPI) control for nonlinear systems with additive process disturbances whose covariance is unknown, spatially varying, and slowly time‑varying. A mismatched disturbance covariance produces a persistent penalty in closed‑loop stability certificates, while online estimation can reduce this penalty as data are collected. We propose a cell‑wise recursive covariance estimator with spatial diffusion and prove a finite‑horizon error bound that separates stochastic‑approximation error, spatial‑smoothing bias, and temporal‑drift effects. The diffusion kernel is chosen to be reversible with respect to the stationary visitation measure, making the diffusion operator dissipative in the weighted Lyapunov analysis. We then substitute the resulting covariance estimate into the MPPI sampling distribution and derive an adaptive stability certificate with an explicit learning penalty. The main result is a payoff theorem: after a computable crossover time, the adaptive controller achieves a strictly tighter certified stability bound than any fixed covariance choice whose mismatch exceeds the residual smoothing and drift allowance. Numerical experiments illustrate the estimator convergence and the resulting stability‑tightening effect.
Authors:Fangxu Yu, Tao Feng, Dehai Min, Lu Cheng, Ge Liu, Tianyi Zhou
Abstract:
Time series reasoning is essential for real‑world problem‑solving. While both Large Language Models (LLMs) and Vision‑Language Models (VLMs) can reason about time‑series data, their capabilities are complementary: LLMs process time series as text sequences and thus preserve exact numerical understanding, but struggle with global patterns, whereas VLMs efficiently capture these patterns by visualizing time series but may lose fine‑grained details. Moreover, models vary significantly in task‑specific expertise and inference costs. Dynamically selecting the most suitable modality and model for each query is therefore crucial, yet challenging because it requires modeling the complex interactions among tasks, queries, modalities, and models, which carry rich contextual signals. To this end, we introduce TSRouter, a graph‑based dynamic routing framework. TSRouter constructs a heterogeneous graph of task, query, modality, and model nodes to contextualize the interactions among query characteristics, modality attributes, and model capabilities. TSRouter formulates routing as a candidate scoring problem, where each modality‑model pair is evaluated based on user‑defined performance‑cost preferences to select the optimal candidate. Comprehensive evaluations on 4 distinct time series reasoning tasks reveal that TSRouter substantially outperforms diverse baselines with 16% to 46% relative improvements. Furthermore, TSRouter demonstrates robust zero‑shot plug‑and‑play generalization to unseen models and novel tasks and preserves high performance while reducing computational overhead through cost‑aware optimization. Our code is available at https://github.com/tianyi‑lab/TSRouter.
Authors:Alex Bercik, Lisa Patrascu, David W. Zingg
Abstract:
We construct optimized summation‑by‑parts (SBP) operators for general function spaces with provably minimal degrees of freedom on open, closed, and half‑open nodal distributions. These operators rely on generalized Gaussian quadrature rules, for which we present an improved algorithm that is flexible, efficient, and provably convergent. In cases where free parameters are available, we further introduce two operator optimization strategies. We test our operators on a handful of numerical examples that contain large or unbounded gradients, in which some a priori knowledge of the solution has been assumed to select an appropriate basis. The novel operators are found to outperform standard polynomial operators by several orders of magnitude in solution accuracy relative to degrees of freedom. Furthermore, our novel operators significantly outperform function‑space SBP operators with equispaced nodal distributions, which require significantly more nodes for the same operator basis. Finally, we demonstrate that the operator optimization procedures are critical to achieving accurate and efficient discretizations, as the standard SBP construction procedure can lead to nullspace‑inconsistent and poorly‑conditioned operators.
Authors:Matthias Tangemann, Benjamin Lo, Zygmunt Pizlo, Kaleem Siddiqi, Dirk B. Walther, Sven Dickinson
Abstract:
Figure‑ground organization in the human visual system relies on several shape‑based cues, including surroundedness, convexity, and symmetry. While these cues have been extensively studied using abstract stimuli, little is known about how they operate under natural conditions or how they arise from the statistics of natural scenes. Deep neural networks offer a promising path forward: a model that relies on the same figure‑ground cues as humans would provide tractable experimental access to the underlying mechanisms. In this study, we evaluate shape‑based figure‑ground organization in Vision Transformers (ViTs), for which prior work has demonstrated the emergence of object‑based grouping. We test 25 ViTs spanning supervised and self‑supervised training objectives, by fitting linear probes to predict figure‑ground assignment from intermediate patch representations using both natural images and controlled artificial stimuli that isolate individual cues. Our results show that ViTs robustly encode surroundedness and convexity, and that probes trained on natural images generalize zero‑shot to artificial stimuli across several models. For symmetry we observe mixed results: the cue is encoded for uniformly colored but not for textured regions. Taken together, our findings demonstrate that Gestalt‑like figure‑ground cues can be learned from natural scene statistics and position ViTs as a compelling model system for studying the computational mechanisms of perceptual organization. Code and data is available at https://github.com/mtangemann/mlvbench
Authors:Aditya Joglekar, Amit Regmi, Kenji Shimada, Levent Burak Kara
Abstract:
Engineering design intent is often communicated through rasterized orthographic drawings. However, downstream workflows inherently require editable and parametrically defined 3D computer‑aided design (CAD) models. To bridge this gap, we introduce Ortho2CAD, a vision‑language model (VLM) specifically designed to translate rasterized orthographic drawings directly into editable CadQuery code, which can then be seamlessly converted into 3D CAD models. To train the model effectively, we utilize supervised fine‑tuning (SFT) for instances where explicit CadQuery code labels already exist, and we apply geometry‑grounded reinforcement learning (RL) to optimize the model in scenarios where ground‑truth labels are absent. To enable learning at scale, we create a pythonOCC‑based drawing generator that renders first‑angle orthographic projections from STEP models, complete with dashed hidden lines and key dimensions. On existing datasets encompassing settings both with and without CadQuery supervision, we generate orthographic drawings and show that our model produces 100% syntactically valid code. Moreover, it achieves a 3D CAD intersection‑over‑union (IoU) accuracy that surpasses all baselines, with an average relative improvement of over 7% compared directly against the next best performing model. We show that leveraging VLMs with SFT and RL techniques can effectively pave the way forward for orthographic drawing to 3D CAD reconstruction. Our implementation is available at https://github.com/AdityaJoglekar/Ortho2CAD.
Authors:Chenjian Gao, Linning Xu, Tianfan Xue
Abstract:
Indoor scene relighting demands photorealism, precise spatial control, and strict multi‑view consistency. While diffusion‑based image editing models enable semantic lighting manipulation via text prompts, enforcing exact 3D light placement often disrupts their generative priors. We propose Lume‑Palette, a progressive framework that leverages semantic lighting priors for spatially controllable multi‑view indoor relighting. The approach decouples relighting into two stages: (1) illumination distillation, which extracts canonical illumination palettes from a pretrained diffusion model to preserve realistic material‑light interactions, and (2) illumination casting, which explicitly maps target spatial lighting conditions defined from coarse 3D geometry. To efficiently handle dense multi‑view and multi‑modal inputs, we introduce an asymmetric multi‑view conditioning strategy that selectively injects essential spatial context. Experiments on diverse synthetic scenes and real‑world scenes demonstrate that Lume‑Palette produces photorealistic, spatially controllable, and multi‑view consistent relighting results. Project Page: https://cjeen.github.io/lumepalette
Authors:Michael Murray, Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Galen Mullins, Harshavardhan Gajarla, Oier Mees, Maya Cakmak, Andrey Kolobov
Abstract:
Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real‑world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large‑scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample‑ and compute‑efficient method for adapting frozen generative robot policies from human interventions in latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse‑time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real‑world bimanual and single‑arm manipulation, adapting both action‑head VLAs and world‑action models from a handful of interventions. FlowDAgger outperforms supervised fine‑tuning and latent‑space RL baselines and preserves pretrained skills on held‑out tasks, offering a practical path for adapting robot foundation models in the real world. Website: https://microsoft.github.io/FlowDAgger
Authors:Dominick Reilly, Qiyu Wu, Hiromi Wakaki, Srijan Das, Yuki Mistufuji
Abstract:
Multimodal Large Language Models (MLLMs) are typically designed under the assumption that all modalities available during training will also be accessible at inference. However, many real‑world settings violate this assumption, requiring models to operate under a privileged modality setting, where auxiliary modalities are available only during training. While these modalities contain valuable information, existing MLLMs largely fail to leverage them effectively, as they treat modalities as interchangeable inputs rather than sources of complementary supervision. We propose Mixture of Probes (MoP), a novel framework that disentangles modality‑specific and modality‑general signals within the MLLM, allowing the model to preserve modality‑dependent structure while learning transferable representations across modalities. At its core, MoP achieves this through a structured probing mechanism that extracts and organizes information from intermediate representations of a shared modality encoder, rather than relying only on final‑layer alignment as done in existing MLLMs. To support this disentanglement, we further introduce MoP Cross‑modal Training (MoP‑X), a training strategy for MoP centered around a probe disentanglement loss that prevents probe collapse and encourages cross‑modal learning. We evaluate MoP across two domains spanning eight tasks and four modalities under a comprehensive evaluation protocol tailored to the privileged modality setting, where each modality is independently treated as the sole input at inference time. MoP consistently outperforms strong MLLM baselines, achieving up to 65% relative improvement, demonstrating that auxiliary modalities, even when unavailable at inference, can provide substantial gains when effectively leveraged during training. Code, model checkpoints, and evaluation protocols will be made available at https://github.com/Sony/MoP.
Authors:Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed, Ruiyang Luo, Nitish Dashora, Richard Li, Pulkit Agrawal, Zhang-Wei Hong
Abstract:
Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers. Standard methods inject stochasticity in the action space, but such jitter only yields rollouts close to the original. Escaping a weak policy often requires global perturbations that action noise cannot produce. Large language models (LLMs) and vision‑language‑action (VLA) models offer a pathway: they condition the policy on a natural language prompt, and since the rollout follows from it, modifying the prompt induces global changes. The challenge is finding prompts that induce useful global changes. With a weak policy that rarely succeeds, reward is too sparse to select on. Our idea is to refine prompts from the rollouts themselves: a vision‑language model (VLM) reasons over the rollout video, diagnoses how the policy responded, and rewrites the prompt to elicit better behavior next time. This procedure realizes posterior sampling, a classical RL exploration framework, at the level of prompts: the VLM maintains an implicit distribution over useful prompts and updates it from observed rollouts. We call this strategy Prompt‑Driven Exploration (PDE). Across manipulation and reasoning tasks, PDE enables RL to learn successful policies even from zero‑reward starts, and improves sample efficiency more broadly. Our website is available at https://xinyunsunshine.github.io/prompt‑rl.
Authors:Tao Lu, Haoyu Wang, Zonghui Wang, Keshen Xiang, Jiaheng Zhang, Wenzhi Chen
Abstract:
With the growing deployment of large language models (LLMs), LLM inference cost has become a key challenge. Pruning techniques that introduce sparsity into weight matrices can accelerate inference. However, maintaining model quality typically limits pruning to moderate unstructured sparsity (around 50%). At these sparsity levels, none of the existing GPU kernels for sparse matrix multiplication (SpMM) can outperform their dense counterparts. This paper proposes an efficient GPU inference method for LLMs with moderate sparsity. We propose a three‑layer matrix storage format comprising: (i) a Sparse‑TC layer enabling sparse tensor cores to accelerate SpMM; (ii) a Slot‑Filling layer using parallel differential distance for matrix compression while supporting low‑cost on‑chip decoding; (iii) a lightweight Residual Layer ensuring correct SpMM computation. Building on this format, we design a SpMM kernel that jointly utilizes sparse tensor cores and CUDA cores. This design enables an efficient execution pipeline and overlaps on‑chip computation with memory access. Evaluations show that our work is the first to outperform dense matrix multiplication on modern GPUs equipped with high‑bandwidth memory (HBM). It achieves up to 1.64x kernel‑level speedup over SpInfer (EuroSys'25, Best paper) and up to 1.41x end‑to‑end speedups over FlashLLM (VLDB'24). Our source code: https://github.com/moui0/cudac.
Authors:Qiheng Sun, Hongwei Zhang, Junxu Liu, Xiaokai Mao, Jinfei Liu, Kui Ren, Haibo Hu
Abstract:
High‑quality data drives machine learning advances across industries. Recognizing the value of data, data transactions are increasingly common, giving rise to many data marketplaces, e.g., AWS Marketplace, Databricks, and Datarade. However, determining the appropriate prices for data products remains a significant challenge due to the unique properties of data products. Traditional pricing methods in economics can be categorized into the cost approach, the income approach, and the sales comparison approach. The cost approach fails in data pricing due to near‑zero marginal cost from data replication, and the income approach fails due to inherently unpredictable data revenue. The sales comparison approach remains viable, yet its application is hindered by the absence of standardized pricing benchmarks for data products across marketplaces. To address this challenge, we introduce \textttDaDaDa, the first dataset for data product pricing, containing metadata for 16,147 data products from 9 major data marketplaces worldwide. \textttDaDaDa enables the training of pricing models, thereby establishing price benchmarks for new data products. In addition, \textttDaDaDa can be utilized for other important tasks in data markets, such as data product classification and retrieval. Experiments and a retrieval prototype demonstrate the effectiveness of \textttDaDaDa for pricing, classification, and retrieval of data products. The dataset and code are available at https://github.com/ZJU‑DIVER/DaDaDa.
Authors:Ziheng Chen, Yue Song, Rui Wang, Xiao-Jun Wu, Nicu Sebe
Abstract:
Manifold‑valued measurements are prevalent in various machine learning tasks. Recent advances have extended Deep Neural Networks (DNNs) to operate on manifolds. These extensions have been accompanied by normalization techniques tailored to different geometries, collectively referred to as Riemannian normalization. However, most existing Riemannian normalization methods are either designed for specific manifolds or fail to effectively normalize manifold‑valued sample distributions. To address these limitations, we propose LieBN, a framework for Riemannian Batch Normalization (RBN) over Lie groups. Our approach leverages the theoretically convenient left‑ and right‑invariant metrics, which naturally exist in every Lie group, and provides theoretical guarantees for controlling the Riemannian mean and variance. We instantiate LieBN across nine distinct geometries: four on the Symmetric Positive Definite (SPD) manifold, one on the group of rotation matrices, and four on the manifold of full‑rank correlation matrices. Notably, among the SPD metrics, we introduce a novel right‑invariant metric and extend three existing Lie group structures via matrix power deformation. Experiments on different manifolds validate the effectiveness of our framework. The code is available at https://github.com/GitZH‑Chen/LieBN.git.
Authors:Kehan Guo, Yili Shen, Yujun Zhou, Yue Huang, Chujie Gao, Shiyi Du, Xiangliang Zhang
Abstract:
The coupling in flow matching ‑‑ the rule pairing noise vectors with data points ‑‑ is typically treated as a computational choice. We show that this coupling can instead serve as an alignment interface: by matching noise and data according to a target molecular property, it embeds controllable structure directly into the learned flow field. Building on this view, we introduce Reward Transport, which uses optimal transport coupling at training time to align a scalar noise‑space coordinate with molecular rewards; at inference, varying this coordinate steers the generated distribution without requiring an oracle, reward model, gradient guidance, or additional computation. In the coupling‑preserving limit, thresholding this coordinate recovers the Cross‑Entropy Method's truncated reward distribution, providing a principled, continuously adjustable distribution‑level control knob. Empirically, on ZINC‑250K and GuacaMol, sweeping the scalar induces monotone control of logP and consistent QED control over its operating range; most tellingly, the same knob produces opposite structural responses for different targets, growing molecules for logP but shrinking them for QED, which rules out a generic size bias. The interface is complementary to classifier‑free guidance and conditional flow matching, while a negative result under epsilon‑prediction diffusion clarifies where coupling‑level alignment is structurally absent. Code: https://github.com/KehanGuo2/reward‑transport
Authors:Jiangwei Ren, Xingyu Jiang, Zijie Song, Wei Xu, Hongkai Lin, Dingkang Liang, Xiang Bai
Abstract:
Estimating 3D geometry in underwater environments presents unique challenges due to light attenuation, scattering, and the absence of large‑scale, high‑quality 3D annotations. Pioneering methods rely on massive dense annotations that are impractical in underwater settings. In this paper, we propose Wat3R, a cross‑domain semi‑supervised learning framework designed to adapt feed‑forward 3D reconstruction models from air to underwater scenes. Uniquely, our method eliminates the need for any annotated underwater data following a teacher‑student architecture, that learns robust geometry representations merely on abundant unlabeled real underwater video footage. We also design a cross‑view consistency loss that leverages geometric cues from other views to compensate for the information degradation in the current view caused by water attenuation and scattering. Furthermore, considering the lack of comprehensive evaluation benchmarks, we construct Water3D, a diverse dataset covering various water bodies and underwater scenarios, designed for geometric task evaluation. Experimental results demonstrate that Wat3R outperforms current state‑of‑the‑art methods in underwater multi‑view depth estimation and point cloud reconstruction. The dataset and code are available at https://github.com/LSXI7/Wat3R .
Authors:Fabio Tosi, Luca Bartolomei, Matteo Poggi, Stefano Mattoccia
Abstract:
Monocular depth estimation has seen remarkable progress through foundation models achieving robust zero‑shot generalization, yet their computational demands place them far beyond the reach of embedded and mobile platforms. Lightweight alternatives exist, but have been developed almost exclusively within single‑domain, self‑supervised paradigms, failing silently under domain shift. We present ZipDepth, a compact monocular depth network that bridges this gap by combining an efficient reparameterizable encoder‑decoder with large‑scale knowledge distillation from a foundation model over a large multi‑domain training set. Comprising just 6.1M parameters, ZipDepth runs at real‑time rates from server GPUs to power‑constrained devices, achieving the best trade‑off between zero‑shot accuracy and deployment efficiency among lightweight models across five benchmarks, taking a significant step towards the accuracy of foundation models with 50x more parameters.
Authors:Cheng-De Fan, Chun-Wei Tuan Mu, Chen-Wei Chang, Chin-Yang Lin, Kun-Ru Wu, Yu-Chee Tseng, Yu-Lun Liu
Abstract:
Recovering high‑quality video from sparse event streams is a challenging task. Regression methods often blur textures, while existing generative models struggle with long‑term stability. We propose LongE2V, a novel approach that leverages pre‑trained video diffusion priors to jointly handle event‑based video reconstruction, prediction, and frame interpolation. By fine‑tuning a foundational video model, our approach achieves high data efficiency and superior perceptual quality. We introduce Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in extremely long sequences. We also propose Reencoding Alignment with Cross Residual Correction to ensure precise bidirectional consistency during frame interpolation. Furthermore, Event Voxel Density Augmentation ensures robustness across varying sensor resolutions. Extensive experiments on real‑world benchmarks demonstrate that LongE2V outperforms state‑of‑the‑art methods across all three tasks, exhibiting exceptional temporal coherence and zero‑shot generalization. Project page: https://cdfan0627.github.io/LongE2V‑page/
Authors:Weijian Chen, Weibo Yao, Yuhang Zhang, Xiaolin Tang, Guo Wang, Weijun Zhang, Xitong Gao, Yihao Chen, Hongde Qin, Lu Qi
Abstract:
Scaling 3D Gaussian Splatting (3DGS) to large outdoor scenes is costly in both data acquisition and computation. Adopting panoramic images with equirectangular projection (ERP) can reduce capture effort via their full 360^\circ field of view, yet the resulting omnipresent visibility invalidates existing partitioning strategies that rely on local camera frustums, causing block‑wise optimization to degenerate into global training. Thus, we propose PanoLOG, a two‑stage coarse‑to‑fine framework equipped with a Geometry and Gradient‑based Partitioning Strategy tailored for large‑scale panoramic 3DGS reconstruction. In the global coarse stage, PanoLOG leverages sky‑sphere modeling and panoramic monocular depth supervision for reliable geometry, while in the refinement stage, G^2PS builds adaptive bounding volumes via parallax‑driven uncertainty and assigns cameras via gradient‑based importance scoring. Furthermore, we construct Pano360, the first benchmark on large‑scale panoramic dataset for outdoor scene reconstruction. Extensive experiments demonstrate that G^2PS achieves state‑of‑the‑art rendering quality while maintaining scalable, block‑parallel training. Our models, training code, and dataset are publicly available.
Authors:Zhekai Chen, Chengqi Duan, Kaiyue Sun, Bohao Li, Yuqing Wang, Manyuan Zhang, Xihui Liu
Abstract:
The rapid development of large language models and multimodal large language models has accelerated the emergence of proactive agents capable of operating everyday tools and assisting users in real‑world environments. However, existing benchmarks struggle to evaluate such agents effectively, as they often rely on sandboxed environments and single‑turn evaluation paradigms. Moreover, their scenario‑based task taxonomies mix multiple model capabilities within the same task category, making it difficult to identify the root causes of agent failures. To address these limitations, we introduce UniClawBench, the first capability‑driven benchmark designed to evaluate proactive agents in dynamic, real‑world settings. UniClawBench is built around five foundational model capabilities: Skill Usage, Exploration, Long‑Context Reasoning, Multimodal Understanding, and Cross‑Platform Coordination. Based on these capabilities, we design 400 bilingual real‑world tasks. Unlike previous benchmarks that rely on static, pre‑recorded answers, our benchmark evaluates agents in live Docker containers using fine‑grained, step‑by‑step completion checkpoints. Furthermore, we design a closed‑loop evaluation strategy comprising an executor agent, a hidden supervisor agent, and a user agent to simulate realistic multi‑turn human feedback without leaking grading criteria. To disentangle base model capabilities from framework‑level design choices, we evaluate state‑of‑the‑art models under multiple agent frameworks. Through comprehensive comparisons across both models and frameworks, we show how base model capabilities and agent framework designs jointly shape performance in real‑world environments. To facilitate future research, we make our benchmark and code publicly available at https://github.com/HKU‑MMLab/UniClawBench.
Authors:Hongyu Liu, Chun Wang, Feng Gao, Xuanhua He, Yue Ma, Ziyu Wan, Yong Zhang, Xiaoming Wei, Qifeng Chen
Abstract:
We propose OPSD‑V, an on‑policy self‑distillation paradigm for post‑training few‑step autoregressive (AR) video diffusion models. Existing few‑step AR video generators can produce long videos with low latency, but still suffer from error accumulation and weakened motion dynamics during long autoregressive rollout. OPSD‑V reduces long‑horizon degradation while preserving the original few‑step inference path. The key idea is to introduce real long‑video data as temporal context during training and use it to provide dense trajectory‑level supervision. Specifically, the student follows the exact inference‑time rollout, generating each chunk conditioned on its own previously generated KV cache. In parallel, the teacher is evaluated at the same student‑visited denoising states, but uses a cleaner AR‑consistent temporal cache in which older history can be replaced by real‑video context. This provides dense denoising‑level corrective targets under on‑policy AR cache dynamics, without changing the sampler, number of denoising steps, or inference‑time cache mechanism. We apply OPSD‑V to representative few‑step AR video models, including Self‑Forcing and LongLive. Experiments show consistent improvements in visual quality, motion dynamics, and VBenchLong scores. A user study with 10 participants comparing 20 video pairs shows that OPSD‑V is preferred over the base models in 66.0% of overall‑preference judgments (82.5% excluding ties).
Authors:Haoran Feng, Ruiyang Zhang, Longyi Zhang, Dizhe Zhang, Lu Qi
Abstract:
In this work, we present Canvas360, a two‑stage framework for in‑context panoramic generation that combines geometry‑aware pretraining with downstream task‑specific fine‑tuning. To address the lack of large‑scale, high‑quality training data tailored to in‑context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high‑quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in‑context generation scenarios. On the modeling side, Canvas360 enhances text‑to‑panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry‑aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in‑context panoramic generation framework that supports diverse downstream tasks via token‑level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama‑specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/
Authors:Yunchao Yao, Zhuxiu Xu, Tianqi Zhang, Zixian Liu, Sikai Li, Zhenyu Wei, Feng Chen, Dihong Huang, Kechang Wan, Chenyang Ma, Shuqi Zhao, Shenghua Gao, Masayoshi Tomizuka, Yi Ma, Mingyu Ding
Abstract:
Building general‑purpose dexterous manipulation policies requires benchmarks that go beyond isolated tasks to systematically evaluate policies across diverse interaction modes, sensory conditions, and robot embodiments. However, existing benchmarks remain limited in task and data diversity, embodiment coverage, or controllable visual variation, hindering studies of cross‑task and cross‑embodiment generalization. We present DexVerse, a large‑scale and modular benchmark for dexterous manipulation. DexVerse includes 100 tasks spanning a broad range of manipulation skills, including object grasping and relocation, articulated‑object interaction, functional tool use, bimanual coordination, non‑prehensile control, contact‑rich behaviors, multi‑goal execution, and long‑horizon multi‑stage task completion. It supports 3 robot arms and 6 dexterous hands, and is extensible to new tasks, assets, and embodiments. To evaluate visuomotor generalization, DexVerse provides configurable visual variations in textures, background, lighting, and camera viewpoints. We further provide a VR‑based teleoperation interface and 3,180 demonstrations with synchronized proprioceptive, RGB, depth, point‑cloud, and state observations. We benchmark representative methods, including Diffusion Policy, DP3, OpenVLA, and π_0.5, across 19 tasks. Results reveal substantial challenges in task generalization and visuomotor robustness, establishing DexVerse as a promising testbed for general‑purpose dexterous manipulation. Project page: https://ycyao216.github.io/DexVerse.site
Authors:Duen Horng Chau, Donghao Ren, Fred Hohman, Dominik Moritz
Abstract:
While UMAP is widely used for exploring high‑dimensional data, typical workflows focus on its lower‑dimensional embedding, largely overlooking the rich k‑nearest‑neighbor (kNN) graph that UMAP constructs internally. This graph encodes the data manifold in its original high‑dimensional space, before the distortion that UMAP's 2D projection introduces. We demonstrate the untapped potential of this internal representation, showing how standard graph algorithms applied to this graph enhance data sensemaking: (1) PageRank identifies representative data points, (2) k‑core decomposition reveals dense core regions versus sparse periphery, and (3) clustering coefficient detects tight‑knit neighborhoods with highly‑similar data points. Through quantitative and qualitative evaluation on MNIST and Fashion MNIST, we show that these graph‑based analyses are not only practical but also competitive with or complementary to purpose‑built methods (e.g., k‑medoids for exemplar selection, HDBSCAN for density‑based clustering).
Authors:Xinyao Li, Xialin He, Runpei Dong, Saurabh Gupta
Abstract:
Keypoint tracking alone is insufficient for object interaction tasks such as sitting on a chair, wiping a board, or pushing furniture, where the robot can reach the correct pose without making meaningful physical contact with the object. We present CONTACTMIMIC, a learning framework that tracks explicit partlevel binary contact commands alongside keypoint trajectories. CONTACTMIMIC is made possible through the use of contact‑following rewards and a trajectory augmentation scheme aimed at breaking the correlations between keypoint trajectories and contact labels. The resulting policy successfully decouples contact behavior from keypoint geometry, and achieves precise physical contact as well as contact‑controllability (produce or suppress contact during deployment as desired). Simulation experiments across 10 diverse human‑object interaction motions confirm that CONTACTMIMIC exhibits contact controllability that enables it to complete manipulation tasks without task‑specific rewards, while also outperforming keypoint‑only trackers on contact‑relevant tasks. Ablations confirm the necessity of the proposed trajectory augmentation scheme and sim2real deployment validates contact controllability in the real world across 5 different motions. Video results are available on https://lixinyao11.github.io/contactmimic‑page/.
Authors:Tomasz Stanczyk, Yuan Gao, Hardik Agarwal, Seongroo Yoon, Tiantao Zhang, Vincent Calcagno, Francois Bremond
Abstract:
Multi‑object tracking (MOT) has achieved strong performance on benchmarks dominated by short video sequences. However, such datasets do not adequately evaluate long‑term identity preservation, where objects must be tracked consistently over extended durations. We introduce WaspMOT, a benchmark designed to address this gap through long‑duration tracking of Trichogramma wasps in controlled ecological experiments. The dataset contains 10 sequences of approximately 12,000 frames each (over 8 minutes at 25 FPS), with dense MOTChallenge annotations and oracle detections to isolate association performance. Unlike existing benchmarks, WaspMOT forms a closed‑set tracking scenario where all individuals remain present throughout the sequence, requiring consistent identity assignment across thousands of frames despite abrupt jumps, occlusions, and highly similar appearance. We establish a benchmark by evaluating five tracking‑by‑detection methods, including ByteTrack, BoT‑SORT, C‑BIoU, OC‑SORT, and McByte, under a unified protocol. Results show that all methods suffer from significant trajectory fragmentation, highlighting the difficulty of long‑term identity preservation even with perfect detections. A simple spatial tracklet stitching baseline consistently improves performance, indicating that substantial gains remain possible. WaspMOT provides a new benchmark for studying long‑term association and reveals limitations of current tracking approaches that are not observable on conventional datasets. The benchmark will be made publicly available at the project repository: https://github.com/tstanczyk95/WaspMOT/ .
Authors:Saw S. Lin, Jyh-Shing Roger Jang
Abstract:
Speculative decoding accelerates LLM inference by drafting several tokens and verifying them in parallel. Block‑diffusion drafters such as DFlash produce a draft block in one pass but model only per‑position marginals, and best‑first tree methods such as DDTree expand candidate trees from those marginals. The released Domino drafter adds a GRU‑based causal correction that makes each draft token distribution path‑dependent, a structure DDTree's factorized formulation cannot represent. We introduce DominoTree, a training‑free best‑first draft tree scored by Domino's conditional, non‑factorized correction along each root‑to‑node path, made practical by restricting the per‑node correction to a candidate top‑M set. On Qwen3‑4B across eight benchmarks, DominoTree reaches up to 6.6x speedup over autoregressive decoding and the highest mean accepted length of any evaluated method, up to 10.7 tokens per round, at every tested temperature. DominoTree constructs its tree with a GPU‑native CUDA‑graph builder that is bit‑identical to a reference Python implementation, so acceptance is unchanged, while keeping per‑round tree construction cheap. With this builder as default, DominoTree improves throughput over the released Domino decoder, the drafter it builds on, at every tested temperature: 9% to 10% overall on Qwen3‑4B and up to 22% on Alpaca. It also outperforms DDTree and CaDDTree at every tested temperature, not only under greedy decoding. On Qwen3‑8B, DominoTree keeps the highest accepted length at every temperature and gives a 24% throughput gain over DDTree at T=0; at higher temperature its edge over DDTree and CaDDTree narrows to a tie and a small loss, while its aggregate gains over DFlash and Domino persist.
Authors:Ziqi Chen, Yingli Zhou, Fangyuan Zhang, Quanqing Xu, Chuanhui Yang, Yixiang Fang
Abstract:
Leveraging large language models (LLMs) to analyze complex documents ‑‑ such as academic papers, technical manuals, and financial reports ‑‑ has emerged as a mainstream and critical task in both research and industry. In practice, users must first filter relevant documents from large collections and then conduct in‑depth analysis (e.g. question answering) over the selected subset, yet existing systems flatten documents into plain‑text chunks, discarding the rich hierarchical structures (sections, tables, figures, equations) and degrading downstream performance. We present DocMaster, a hierarchical structure‑aware document analysis system. DocMaster parses documents into hierarchical document trees preserving original layouts and constructs a structure‑aware semantic index that enables accurate document filtering and in‑depth analysis. We demonstrate DocMaster through an interactive web interface that enables users to upload document collections, construct tree‑based and multi‑view semantic indices, filter relevant documents via natural‑language conditions, and perform follow‑up question answering over the filtered results. The source code, data, and demo are available at https://doc‑master.github.io/.
Authors:Jacob Chalk, Saptarshi Sinha, Dima Damen, Yannis Kalantidis, Diane Larlus
Abstract:
The recently established 'Out of Sight, Not out of Mind' (OSNOM) task for egocentric videos focuses on tracking objects that are moved by the camera wearer, online, maintaining knowledge of instance locations throughout the video even when they leave the field of view or become heavily occluded. In this paper, we propose the first learning‑based solution to the OSNOM task: Whareformer, a transformer‑based model with two components: an updatable memory of established tracks and a track assignment module that associates observations with existing tracks in a feed‑forward manner. Whareformer jointly reasons over evolving object appearance (what) and updated 3D location (where), and employs a dedicated New Track token to reason about novel objects. Thanks to its design choices of using relative distances and evolving track representations, Whareformer is trained on a small set of 56 videos but achieves SOTA performance on 260 long test videos from three datasets: EPIC‑KITCHENS‑100 (unseen videos), IT3DEgo, and HD‑EPIC, with significant absolute improvements over prior work.
Authors:Matteo Spanio, Antonio Rodà
Abstract:
Semantic audio applications increasingly require controllable generation on commodity and embedded hardware rather than through framework‑heavy datacenter stacks. We present aria, a dependency‑free native runtime that runs the complete text‑to‑music pipeline of Stable Audio~3 (SA3) on ordinary GPUs, CPU‑only machines, and a Raspberry~Pi~5, with no Python or deep‑learning framework underneath. Our main contribution is a study of quantization: running the model at lower numerical precision to fit tight memory budgets, saving memory in place rather than adding to it. Because the runtime owns every internal tensor, it also exposes activation steering, a low‑cost way to steer what the model generates. We judge the quality cost with three independent measures of the output (prompt adherence, overall audio quality, taste preservation), each compared against the ordinary variation between random seeds. Eight‑bit precision shows no measurable quality loss on any measure while sharply cutting memory, and it is the fastest mode on the GPU; four‑bit adds a small, bounded cost but shrinks the footprint enough to run the 1.2‑billion‑parameter model on an 8\,GB Pi. Against the official implementation, aria matches or exceeds generation speed and starts about seven times faster. A case study of the steering interface generates music carrying taste associations (\emphsonic seasoning), with genuine but bounded control for a subset of attributes. These results make a compact, quantized runtime with built‑in control a practical basis for on‑device semantic audio in Internet‑of‑Sounds settings. The aria runtime is released at https://github.com/matteospanio/aria.
Authors:Arav Gupta, Nivedan Yakolli, Avinash Gautam
Abstract:
Most cooperative Vision‑Language Navigation (VLN) methods assume unlimited communication, not considering real‑world applications where bandwidth is restricted and information efficiency is critical. We introduce bandwidth‑constrained cooperative VLN and propose hindsight gating: a lightweight supervised gate that labels communication‑critical steps post‑hoc from navigation failures, avoiding the high variance of REINFORCE. Contrary to the intuition that agents should communicate when uncertain, we observe a consistent counter‑intuitive pattern: trained gates fire predominantly in early episode steps and more often when agents are confident, across all budget levels (B \in \1,3,5\). We explain this through recurrent hidden‑state alignment: early communication injects grounded trajectory representations that persist and compound through subsequent Gated Recurrent Unit (GRU) updates, achieving +0.072 cumulative alignment gain with B=3 transmissions, approaching unconstrained communication (+0.078) at 260% greater alignment efficiency than random gating (+0.020) and 320% greater efficiency than entropy‑based gating (+0.017). Our results establish a new communication regime for bandwidth‑limited embodied agents: synchronise representations early, navigate independently later. Our codebase is available at: https://github.com/AravG13/bandwidth‑constrained‑cooperative‑vln
Authors:Feng Wang, Canmiao Fu, Zhipeng Huang, Chen Li, Jing Lyu, Ge Li
Abstract:
Recent unified multimodal models show a single architecture can jointly perform vision/language understanding and image generation/editing. However, they repeatedly feed all historical visual and textual inputs into a shared context window, limiting long‑horizon multimodal dialogue due to visual token explosion and unreliable cross‑turn referencing. We propose a Cognitive‑structured Multimodal Agent that externalizes visual information into an Episodic Visual Memory and selectively reactivates relevant episodes during reasoning. The agent consists of a Perceptual Abstraction Engine for structured visual abstraction, a Cognitive Retrieval Engine for cross‑turn memory retrieval, and a Multimodal Executive Controller for autonomous task inference and action planning. To address the lack of turn‑level retrieval supervision in existing datasets, we develop a Unified Scenario Engine that programmatically generates structured multi‑turn conversations with fine‑grained retrieval annotations, enabling reinforcement learning to optimize abstraction and retrieval policies. We also construct a long‑horizon visual‑dialogue benchmark stratified by difficulty to evaluate episodic visual recall. Our 8B agent achieves 91.4% retrieval accuracy over 20‑turn sessions, surpassing 32B baselines by +8.2% while nearly halving per‑turn inference time (23.1s ‑> 12.7s). We further present the Cognitive‑structured Multimodal Agent Harness (CMA‑Harness), a tool‑augmented deployment of the same cognitive structure integrating persistent multimodal memory, web access, image generation/editing/composition tools, and OpenAI‑compatible serving. Structured memory and modular decision‑making offer a more scalable, efficient paradigm for long‑horizon multimodal agents than monolithic parameter scaling. Code: https://github.com/caseclose/cma‑harness ; Project page: https://caseclose.github.io/cma‑harness/
Authors:Jiewen Deng, Hangchen Liu, Junchen Li, Boyuan Zhang, Renhe Jiang
Abstract:
Multi‑modality transportation refers to urban systems composed of multiple transportation modes, such as traffic flow and public transit, whose dynamics are coupled by shared temporal patterns. Accurate multi‑modality transportation forecasting remains challenging because (1) different modalities exhibit distinct spectral characteristics and (2) interact unevenly across frequencies, whereas most existing methods operate primarily in the time domain or rely on coarse feature fusion. To address these limitations, we propose a lightweight yet effective Frequency‑Domain Multi‑Modality modeling (FreMo) that explicitly exploits the frequency domain to enable adaptive and selective cross‑modality synergy. FreMo disentangles modality‑wise spectral refinement from cross‑modality synergy and supports plug‑and‑play integration with general time series backbones. Specifically, FreMo introduces a Modality‑Wise Frequency Filter (MFF) to adaptively refine spectral components within each modality, emphasizing informative frequencies while suppressing noise. FreMo further incorporates a Frequency‑Guided Synergy Integrator (FSI) that selectively aggregates information across modalities based on their relative contribution at each frequency, facilitating effective cross‑modality knowledge sharing while mitigating negative transfer. Extensive experiments on real‑world datasets show that FreMo consistently outperforms state‑of‑the‑art baselines, with superior performance and generalization across diverse forecasting scenarios. The code is available at https://github.com/beginner‑sketch/FreMo.
Authors:Jorge Ignacio Perez, Hwaai Kang Kee, Lucas Rassbach
Abstract:
Determining agricultural potential is fundamental to sustainable land management and agricultural planning. Remote sensing data is increasingly valuable as an avenue for agricultural potential due to the cost of traditional methods (surveys, in‑situ measurements, soil testing, etc). ImageCLEF AI4Agri 2026: Subtask 1 is concerned with the prediction of viticulture potential in Southern France. The DS@GT ARC's submission for Subtask 1 introduces an ensemble of U‑Net and a Geospatial Foundation Model (Prithvi‑2.0). Our best model achieved a \pm1 accuracy of 68.32 on the leaderboard, ranking 2nd among 7 teams. The implementation for this work is publicly available at https://github.com/dsgt‑arc/imageclef‑ai4agri‑2026 .
Authors:Baoyu Li, Xinchen Yin, Mengying Lin, Yixin Zhang, Danfei Xu
Abstract:
Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics, with non‑transferable factors like human morphology, head motion, and behavioral style. We study whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions, but also how the scene evolves. The central question is what world representation best enables human‑to‑robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent‑invariant physical effects, and separate camera motion from environment change. We introduce EgoWAM, a controlled human‑robot co‑training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real‑world bimanual tasks, WAM co‑training scales more effectively with in‑the‑wild egocentric human data than behavior cloning. Pixel‑based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out‑of‑distribution object and scene generalization by up to 4x, and 3D flow improves in‑domain performance by 20‑30%. More details: https://gatech‑rl2.github.io/egowam.github.io
Authors:Pengjie Wang, Linger Deng, Zujia Zhang, Shaojie Zhang, Zhenbo Luo, Pei Fu, Jian Luan, Xiang Bai, Yuliang Liu
Abstract:
Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visual state as a full image. This full‑image generation paradigm introduces substantial visual‑token redundancy and dilutes supervision on sparse yet reasoning‑critical state transitions. We propose DeltaV, a ULMM that replaces full‑image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update tokens that capture the visual changes across reasoning steps, avoiding repeated modeling of unchanged content. To align the token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable reasoning, we further construct StructCoT, a large‑scale interleaved multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual‑update paradigm reduces newly generated visual tokens by 55.6% on average without compromising reconstruction fidelity, and improves multimodal reasoning by 3.3% over full‑image generation. Trained with StructCoT and large‑scale multimodal data, DeltaV‑2B further outperforms substantially larger open‑source models by 8.4% on in‑domain multimodal reasoning evaluations and surpasses the comparable‑scale Qwen3‑VL‑2B by 5.9% on external multimodal reasoning and understanding benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie‑W/DeltaV.
Authors:Tianyi Song, Sierra Bonilla, Xinwei Ju, Evangelos Mazomenos, Danail Stoyanov, Adam Schmidt, Omid Mohareri, Sophia Bano, Francisco Vasconcelos
Abstract:
Gaussian splatting is the current state‑of‑the‑art for dense, deformable 3D anatomy reconstruction in robot‑assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track‑anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non‑SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.
Authors:Ali Motahharynia, Mohammadreza Ghaffarzadeh-Esfahani, Mahsa Sheikholeslami, Navid Mazrouei, Matin Irajpour, Yousof Gheisari, Hajar Sirous
Abstract:
Current computational approaches for drug design typically focus on generating molecules conditioned on specific targets or general molecular properties, often neglecting the influence of disease context on target behavior and therapeutic outcomes. To address this gap, we introduce DrugGen‑2, a novel generative model that designs small molecules conditioned on both disease ontology and target protein sequences. DrugGen‑2 was developed by fine‑tuning a pre‑trained GPT‑2 model on a curated dataset of approved drugs linked to their diseases and targets, using a two‑step strategy of supervised fine‑tuning followed by reinforcement learning via group relative policy optimization (GRPO). This process was guided by reward functions optimizing for chemical validity, novelty, diversity, and high predicted binding affinity. When evaluated on five protein targets relevant to diabetic nephropathy, DrugGen‑2 significantly outperformed baseline models (DrugGPT and DrugGen). It demonstrated a superior capacity to generate unique molecules, exhibited greater structural similarity to approved drugs, and achieved improved predicted binding affinities across all targets. Molecular docking analyses further supported these findings, identifying candidate ligands with strong binding potential, including compounds with predicted affinities (‑9.917, ‑9.485, and ‑9.367) exceeding those of reference drugs such as enalapril for angiotensin‑converting enzyme (‑8.283). By integrating disease‑specific context into molecular generation, DrugGen‑2 advances AI‑assisted drug discovery, offering a powerful tool for de novo design and drug repurposing that accounts for the complex interplay between diseases and molecular targets.
Authors:Maximilian Woehrer
Abstract:
Quantifying how mirror‑symmetric an image is about a given axis (symmetry scoring) underpins applications from visual aesthetics to medical imaging, yet proposed scoring methods have never been compared on a common, statistically grounded protocol. We benchmark 13 scoring methods (nine collected from literature, four introduced here) spanning from classical features to frozen deep features, across four single‑axis and five multi‑axis datasets under a reflection‑exact protocol with a chance‑anchored, significance‑tested discrimination skill. Deep backbones perform best on single‑axis and harder multi‑axis protocols. However, a classical histogram‑of‑oriented‑gradients (HOG) descriptor trails the best frozen‑network readout by a small (but significant) margin, is not statistically separable from the runner‑up (a CNN‑filter measure), and runs ~300x faster on CPU. Our results show that discrimination concentrates in mid‑scale oriented features, where deep backbones peak at a low or mid stage, and HOG peaks at a mid cell size. Among existing methods, frozen deep features thus offer little over a tuned classical descriptor for measuring symmetry; whether task‑trained deep scorers can do better remains open. We release the scorers and harness in imgsym, an open toolkit for image symmetry detection and measurement.
Authors:Amir Asiaee
Abstract:
Mechanistic interpretability often evaluates explanations by intervening on a model: swapping hidden states, patching activations, ablating components, or comparing a compressed model to the original one. These experiments are usually summarized by a point estimate, even though the evaluation may be monitored while it runs or adapted toward suspected failures. This makes it hard to tell whether a reported fidelity or patching effect is a stable causal claim or a consequence of finite sampling and evaluation choices. We introduce Certified Interventional Fidelity (CIF), a statistical layer for interventional interpretability evaluations. CIF first writes the quantity being reported as a causal estimand: an expectation of a bounded score over a stated input distribution and a stated intervention distribution. It then provides confidence intervals and anytime‑valid confidence sequences for this estimand, including under adaptive intervention sampling via bounded mixture importance weighting. We instantiate CIF with Hoeffding‑style sequences and variance‑adaptive betting sequences, the latter reducing certification cost by 10‑30x in our experiments. On MNIST abstractions and GPT‑2 Small IOI circuits, CIF certifies high‑fidelity claims, shows when apparent method differences are not statistically supported, and makes sensitivity to the intervention distribution explicit.
Authors:Chenxi Wang, Ying Feng, Hongjie Fang, Shangning Xia, Lixin Yang, Chuan Wen, Cewu Lu
Abstract:
Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retargeting, which maps operator hand motions to feasible and intuitive robot hand motions. Existing methods often require hand‑crafted objectives, precise calibration, or global shape matching between human and robot hand spaces, making them sensitive to hand‑specific tuning and less reliable across different dexterous hands. We propose AnyDexRT, a calibration‑free retargeting method for intuitive dexterous teleoperation across human‑like dexterous hands. AnyDexRT combines self‑supervised fingertip correspondence learning with few‑shot human guidance to anchor the mapping in task‑relevant regions, and further refines pinch‑related poses using a contact classifier. Experiments on diverse dexterous hands and real‑world teleoperation tasks show that AnyDexRT improves retargeting quality, reduces manual tuning, and provides more intuitive and efficient control than prior retargeting methods. Project website: https://chenxi‑wang.github.io/projects/anydexrt
Authors:Noah Jaitner, Kandice Tanner, Ingolf Sack, Hossein S. Aghamiry
Abstract:
Background and Objective: Quantitative analysis of cell dynamics is central to modern biological research, providing critical insights into immune cell interactions, disease progression, and drug mechanisms. Automated cell tracking in time‑lapse microscopy remains challenging due to noise, morphological variations, overlapping cells, and dynamic events such as divisions and fusions. Methods: We present ARGUS, a framework for Accelerated, Robust, General, and Unsupervised Cell Tracking Solutions. ARGUS combines adaptive cell detection, dense Farneback optical‑flow prediction, frame‑to‑frame linear assignment, and a sequence‑level tracklet‑refinement step that reconnects trajectory fragments across short temporal gaps. Results: On publicly available Cell Tracking Challenge datasets, ARGUS achieved detection accuracy of 0.905‑0.971 and tracking accuracy of 0.897‑0.964, with runtimes within 1 minute (5‑6 seconds for 3 frames). Conclusions: ARGUS is a modular, interpretable framework that can be adapted to different imaging modalities and biological applications without training data or GPU infrastructure. The implementation is publicly available at https://github.com/Gitinc/argus
Authors:Shaoliang Yang, Jun Wang, Yunsheng Wang
Abstract:
Remaining useful life (RUL) estimates support reliability and maintenance decisions only if both point accuracy and prediction intervals remain trustworthy when operating conditions change. Convenient mixed splits can hide that failure. This paper studies the question on a documented 10‑bearing PHME subset with time‑varying load and speed. Derived load‑speed regimes define the held‑out evaluation units, while models receive only measured load and speed as context. A calibrated predictive‑representation model fuses raw vibration windows, engineered descriptors, and operating context, then forms intervals by empirical residual calibration. Under strict train/validation/calibration/test separation, the model reaches normalized MAE 0.1477, empirical 90% coverage 0.900, and retrospective absolute‑step MAE 285.26; a 400‑tree random forest reaches 0.1538, 0.871, and 294.57. The results do not show uniform dominance: conditional diagnostics expose non‑uniform reliability, including 0.666 coverage in a low‑load/high‑speed cell, and a post‑hoc pooled regime‑conditioned residual diagnostic raises that cell to 0.941 only as motivation for future pre‑specified conditional calibration. Stress tests further identify raw‑channel loss as the largest tested reliability failure mode. The contribution is therefore a bounded reliability‑evaluation protocol for the processed 10‑bearing subset, with conditional undercoverage and raw‑channel loss reported explicitly as failure modes rather than deployment guarantees.
Authors:Yuxiang Feng, Juncheng Wang, Chao Xu, Wenlong Hou, Huihan Wang, Yijie Qian, Yang Liu, Baigui Sun, Yong Liu, Shujun Wan
Abstract:
Forecasting the future anatomy of slow‑evolving neurodegenerative diseases could enable earlier, more targeted intervention and improve clinical trial design, but it remains challenging because true progression signals are subtle in longitudinal MRI. In this low‑signal regime, transferring modern generative sequence models directly is unreliable: training is dominated by stable baseline anatomy and confounded by dense, sample‑specific nuisance variation. We first provide a theoretical analysis that explains these failures through two modes. Identity collapse occurs when optimization is driven toward reproducing the current anatomy, which prevents the model from learning faint temporal change. The continuous interpolation trap arises when standard smooth networks cannot separate localized biological drift from pervasive noise, which leads to spurious changes that diffuse across the volume. To address both issues, we propose Latent Drift, a progressive generative framework that learns change in a compressed semantic representation rather than synthesizing full‑resolution anatomy. This design removes pixel‑level identity from the prediction target and concentrates model capacity on progression‑relevant dynamics. We further apply Finite Scalar Quantization to the learned change representation, which suppresses small, high‑frequency nuisance fluctuations while preserving consistent structural drift. Experiments on longitudinal 3D brain MRI show that Latent Drift improves patient‑specific neuro‑forecasting over diffusion and autoregressive transformer baselines across generative fidelity and clinically relevant evaluation metrics. Project page: \hrefhttps://cutepkq.github.io/latent‑drifthttps://cutepkq.github.io/latent‑drift.
Authors:In-Hwan Jin, Hyeongju Mun, Joonsoo Kim, Kugjin Yun, Kyeongbo Kong
Abstract:
Dynamic scene reconstruction remains challenging due to the heterogeneous and spatially varying nature of real‑world motion. Although recent 3D Gaussian Splatting methods have introduced diverse deformation formulations for dynamic novel view synthesis, each method typically relies on a single deformation model within its representation, which limits robustness across diverse dynamic scenarios. In this work, we study a fundamental problem‑multi‑deformation modeling for dynamic 3D Gaussian representations‑under two distinct integration constraints that differ in when and how multiple deformation experts interact during training. From a Mixture‑of‑Experts (MoE) perspective, we view multi‑deformation modeling as the problem of combining multiple specialized deformation models within a unified 3D representation. We first introduce Mixture of Deformation Experts (MoDE), which integrates multiple deformation experts directly into the deformable Gaussian Splatting pipeline through joint optimization. In MoDE, experts operate on a shared canonical Gaussian representation, enabling multi‑deformation modeling without introducing additional training stages or modifying the original optimization schedule. In contrast, we further present Mixture of Experts for Dynamic Gaussian Splatting (MoE‑GS) under a different integration constraint, where deformation experts are optimized independently and combined through a separate routing stage. As a result, expert interaction occurs over non‑canonical Gaussian representations after individual optimization. Together, these two approaches provide alternative strategies for multi‑deformation modeling, clarifying how integration constraints shape the design and behavior of deformation experts in dynamic 3D Gaussian representations. Our code is available at: https://github.com/cvsp‑lab/MoE‑GS‑studio.
Authors:Renato Cordeiro de Amorim
Abstract:
The k‑means++ algorithm is commonly restarted multiple times to avoid poor local optima, yet the number of restarts is almost always chosen arbitrarily and applied uniformly regardless of data set difficulty. This undermines any comparison relying on such a choice and wastes computation on easy data sets while potentially under‑serving hard ones. We introduce GTRC, a restart criterion combining a Good‑Turing estimate, a proven unconditional bound, and a confidence‑based bound on the probability that a further restart would improve on the current result, stopping once this probability falls below a user‑specified tolerance \varepsilon. Across 36 data sets, GTRC reached clustering quality competitive with well‑chosen fixed restart counts, while the number of restarts used varied considerably and appropriately with data set difficulty, governed by an interpretable, data‑dependent signal rather than a fixed rule. GTRC offers a principled and reportable alternative to fixing the number of k‑means++ restarts in advance. Software:https://github.com/RCdeAmorim/Good‑Turing‑Restart‑Criterion.
Authors:Hyeonseop Song, Seokhun Choi, Hoseok Do
Abstract:
Large‑vocabulary instance segmentation is constrained by long‑tailed category distributions and fine‑grained inter‑class ambiguity. While data synthesis offers a promising alternative, current paradigms have complementary limitations: text‑to‑image (T2I) methods inherit noisy pseudo‑labels and struggle on rare classes, whereas copy‑paste methods compromise contextual realism. To address these issues, we propose a hybrid pipeline coupling T2I generation with context‑aware image‑to‑image (I2I) editing. The T2I branch provides broad category and scene diversity, while a teacher‑student scheme ensures label reliability by selectively retaining only prompt‑specified categories. To strengthen supervision for rare classes, we introduce VRAIN (Verified Rare‑class Augmentation via INstructed editing), a novel I2I editor. VRAIN inserts high‑confidence instances at semantically appropriate locations within in‑the‑wild scenes, yielding semantically coherent and visually natural edits that reduce domain gaps and enable targeted augmentation. On the LVIS benchmark, our method surpasses existing baselines, improving overall AP by up to +4.0 points and rare‑class AP by up to +9.5 points, while scaling effectively with backbone capacity. Our project page is available at https://seokhunchoi.github.io/TMI
Authors:Qishun Wang, Yapeng Li, Bin Luo, Zhengzheng Tu, Chenglong Li
Abstract:
RGB‑Thermal (RGBT) Video Object Detection (VOD) has gained significant traction due to its ability to overcome the limitations of conventional RGB‑based VOD under challenging conditions. However, spatial misalignment commonly exists between RGBT image pairs. To address this, we propose a Dual‑Correlation Hypergraph Network (DHNet) that captures high‑dimensional complementary information by explicitly modeling two types of correlations: temporal correlation across consecutive frames and spatial correlation from cross‑modal features. Specifically, we first design a Patch‑based Spatial Alignment Module (PSAM) to sequentially align the multimodal features at the local region level. Subsequently, we introduce a Dual Hypergraph Fusion Module (DHFM), which constructs separate temporal and multimodal hypergraphs to enhance object discriminability through dual‑correlation learning. Furthermore, the field currently lacks a large‑scale, scene‑diverse benchmark dataset for comprehensive evaluation. To address this gap, we construct DVT‑VOD1000, a large‑scale RGBT VOD dataset containing 1,000 video sequences with 103,464 RGBT image pairs. The dataset covers diverse scenarios, including campuses, parks, transportation, rural areas, night scenes, rain, and snow. Comprehensive experiments on VT‑VOD50 and our DVT‑VOD1000 demonstrate that DHNet achieves state‑of‑the‑art detection accuracy. The dataset and source code will be made publicly available on https://github.com/tzz‑ahu/ to support academic research.
Authors:Qi Lyu, Baicheng Liu, Xudong Wang, Jiahua Dong, Lianqing Liu, Zhi Han
Abstract:
Vision‑language‑action (VLA) models aim to map multimodal inputs to robot actions. However, most existing approaches struggle to cover complex dynamic scenarios due to treating all visual tokens uniformly and reasoning with human‑selected factors, which lack mechanisms to emphasize task‑critical evidence and ignore underlying factors. To address this issue, we propose LEEVLA, a VLA architecture for seeing what matters in Latent Environment Evolution that explicitly guides the model toward informative regions while preserving the structured evolution of latent world representations. To identify salient and instruction‑relevant regions, we introduce drift‑guided dynamic prioritization (DGDP), which combines dynamic position prioritization (DPP) with semantic drift guidance (SDG) to guide the VLA agent where to attend during training. On top of this, we introduce structured feature flow generation (SFFG), which models how these prioritized features should evolve in latent space via prototype‑to‑periphery (P2P) prediction, and a mutual‑neighborhood contrastive (MC) loss to maintain topological consistency among neighborhoods. Together, DGDP and SFFG form a task‑aware "where‑how" training framework. Extensive experiments on VLA benchmarks show that LEEVLA consistently outperforms prior methods, confirming that explicit task‑evidence guidance and structured latent reasoning are both crucial for scalable VLA. Our code is available at https://github.com/LyuQi127/LEEVLA.
Authors:Amir Asiaee, Kaveh Aryan
Abstract:
Workload‑based differentially private (DP) synthetic data methods privately measure aggregate queries and post‑process the noisy answers into synthetic records. Generic workloads can achieve strong distributional fidelity, but causal estimands such as the average treatment effect (ATE) depend on treatment‑arm balance and outcome moments that generic marginals need not preserve. We propose causal workloads: DP query sets designed around the orthogonal moments used by doubly robust causal estimators. The released workload can be used directly by stable moment‑map estimators or reconstructed by maximum‑entropy calibration into reusable synthetic data; our theory decomposes ATE error into sampling, privacy, workload‑approximation, Monte Carlo, and calibration terms. We also introduce Causal‑AIM, an adaptive workload selector, and a noise‑aware multiple‑imputation (NA+MI) procedure for confidence intervals from DP synthetic data. Because the workload is released once, the same DP synthetic table can support ATE, ATT, and subgroup analyses without additional privacy spending. Empirically, causal workloads are most useful at strict privacy budgets and for calibrated uncertainty, while generic workloads often retain an advantage for point RMSE as privacy relaxes. The broader lesson is a tradeoff: distributional fidelity can help point accuracy, but valid causal inference requires preserving causal moments and propagating DP noise rather than treating synthetic rows as real.
Authors:Guanchen Liu, Hongyang Du, Kaibin Huang
Abstract:
Inference‑time scaling has emerged as an effective approach for enhancing the capabilities of Large Language Models (LLMs), addressing the growing demand for stronger reasoning without increasing model size. This novel form of LLM scaling comprises two representative approaches: explicit reasoning, which generates intermediate chain‑of‑thought tokens during an explicit thinking phase, and implicit reasoning, which iteratively updates hidden states in the latent space without producing explicit outputs. Despite their effectiveness, both paradigms incur substantial computational and memory overhead, raising challenges for deployment on resource‑constrained edge devices. To address these issues, we propose a Mobile Reasoning‑as‑aService (MORES) framework that treats reasoning as a computational service accessible to edge devices over wireless networks. Focusing on implicit reasoning, we leverage its recursive structure to partition hiddenstate updates between edge devices and servers, enabling cooperative inference that allows devices to access additional cloud computation on demand. To optimize long‑term performance, we formulate a joint computation and communication scheduling problem and solve it using a semantic Mixture‑of‑Experts (MoE)‑based Deep Reinforcement Learning (DRL) algorithm to address heterogeneity in wireless conditions and task demands. The agent adaptively allocates resources by adjusting the number of recurrent steps and the transmission pruning rate, while a semantic router enables high‑speed gating for real‑time expert selection. Experimental results show that the proposed method achieves an approximately 18% improvement in system throughput over the baseline Soft Actor‑Critic (SAC) algorithm. Our code is available at https://github.com/NICE‑HKU/MORES.
Authors:Hogyun Kim, Jiwon Choi, Jungwoo Lee, Younggun Cho
Abstract:
While global localization using spinning radar has gained attention for its robustness to adverse weather and challenging environments, many studies have focused on individual components such as place recognition or pose estimation. In this paper, we take a holistic view of radar sensor‑based global localization and present RadLoc, a fast, robust, and lightweight end‑to‑end pipeline from place recognition to 3‑DoF pose estimation. RadLoc accelerates pre‑processing using 1D CA‑CFAR filtering and leverages the near‑range dominance in spinning radar images to design a compact descriptor and an efficient hierarchical coarse‑to‑fine retrieval strategy. Moreover, coupled with phase correlation‑based 3‑DoF pose estimation, it forms a versatile global localization module applicable to SLAM and multi‑session SLAM systems. Extensive experiments on 15 sequences across 5 datasets demonstrate that RadLoc achieves robust performance while maintaining the smallest descriptor size and fastest retrieval time among state‑of‑the‑art approaches. The supplementary materials are available at https://sparolab.github.io/research/radloc/.
Authors:Chaewon Lee, BeomJun Shim, Kwang Pyo Choi, Chang-Su Kim
Abstract:
We propose contrastive order learning (ConOrd), a contrastive learning framework for ordinal regression that integrates the strengths of contrastive learning and order learning. While contrastive learning effectively leverages all samples in a batch, it typically ignores the inherent ordering among rank labels. Conversely, order learning explicitly models label ordinality but often relies on local, margin‑based comparisons, limiting its ability to capture global ordinal structure. ConOrd addresses these limitations by introducing a contrastive order loss with soft affinity and disparity weights based on rank differences, enabling fine‑grained modeling of ordinal relationships across all sample pairs within a batch. Extensive experiments on a range of ordinal regression tasks, including facial age estimation, blind image quality assessment, and blind video quality assessment, demonstrate that ConOrd consistently achieves state‑of‑the‑art performance and generalizes well across diverse ordinal regression scenarios. The source code is available at https://github.com/cwlee00/ConOrd.
Authors:Chaewon Lee, Seon-Ho Lee, Chang-Su Kim
Abstract:
Rank estimation under label noise poses a fundamental challenge, as ordinal annotations often exhibit structured uncertainty rather than simple label corruption. In this paper, we reformulate rank estimation with noisy ordinal labels as a stochastic ordering problem, in which each instance is inherently associated with multiple plausible ranks instead of a single deterministic label. Based on this view, we propose stochastic order learning (SOL), a learning framework that captures ordinal label uncertainty and learns an embedding space through two complementary objectives: a discriminative loss that structures instance‑‑centroid interactions and a stochastic order loss that enforces probabilistic ordering relations between instances. Extensive experiments across diverse datasets demonstrate that SOL enables reliable rank estimation under various types and levels of label noise. The source code is available at https://github.com/cwlee00/SOL.
Authors:Linli Shi, Ruijun Zhang, Ziyun Wang
Abstract:
Event cameras offer microsecond temporal resolution, low latency, and high dynamic range, making them attractive for robotics. However, labeled event‑camera data for a specific robot and scene is scarce and expensive to collect, which slows the development of event‑based perception and control. We present EVIS: a physics‑grounded event camera plugin for NVIDIA Isaac Sim that generates high‑rate, fully labeled event streams directly inside a physics simulator. The plugin implements a faithful log‑intensity contrast event model with per‑pixel asynchronous reference updates; it migrates from a normal RGB camera with few changes and integrates into any Isaac Sim / Isaac Lab scene, inheriting the simulator's physics and frame‑perfect ground truth. It is fully configurable, and offers an interpolation option that renders only sparse keyframes and synthesizes the in‑between frames through bidirectional motion‑vector warping, making real‑time generation on a single GPU possible. Optional sensor noise and motion blur further narrow the gap to real cameras. The generated streams are directly usable by pretrained event networks for downstream tasks. Code repository: https://github.com/spikelab‑jhu/isaac‑sim‑event‑camera‑plugin
Authors:Jim Dai, Zhanhao Zhang
Abstract:
The stationary distribution of reflected Brownian motion (RBM) plays an important role in the analysis of high‑dimensional stochastic systems, yet closed‑form solutions are known only for a few special cases. Computing important performance metrics, such as tail probabilities, is even more intractable, despite their practical relevance. In this paper, we develop a deep learning approach that accurately and efficiently learns the Laplace transform of high‑dimensional RBMs based on the basic adjoint relationship (BAR). Our framework combines a careful design of the loss function, training data sampling procedure, and neural network architecture. We evaluate the proposed method on RBM instances with known ground‑truth tail probabilities and demonstrate near‑perfect prediction in high‑dimensional settings, highlighting its potential as a general tool for analyzing stochastic systems beyond analytically tractable regimes. Our code can be found at https://github.com/zhangz73/NN4MGF.
Authors:Matt Y. Cheung, Ashok Veeraraghavan, Guha Balakrishnan
Abstract:
Radiomic features derived from medical images and segmentation masks are used to support decision making in clinical imaging pipelines. In practice, these features are often computed from predicted masks, but segmentation models can be overconfident or poorly calibrated, making derived measurements appear more reliable than they are. Conformal prediction (CP) provides distribution‑free prediction intervals with finite‑sample marginal coverage guarantees, but black‑box intervals for segmentation‑derived radiomics can be inefficient because they ignore test‑time information about image appearance, mask geometry, and segmentation uncertainty. We propose ConRad, a conformal framework for scalar radiomic targets that uses covariates derived from the predicted mask, input image, predicted radiomics, and boundary uncertainty to construct adaptive intervals while maintaining coverage. Across five 2D medical imaging datasets and 171 retained radiomic targets, we show that ConRad improves feature‑level efficiency compared to baselines while maintaining near‑nominal empirical coverage. Ablation results further indicate that segmentation boundary uncertainty features are the largest contributors to interval efficiency.
Authors:Ao Hong, Lehang Wang, Zhirun Yue, Mingxin Wang, Zihan Wang, Houde Liu
Abstract:
Aspect Sentiment Triplet Extraction (ASTE) requires jointly identifying (aspect, opinion, sentiment) triples from a given review sentence. While large language models (LLMs) achieve strong zero‑shot performance on many NLP benchmarks, their effectiveness on ASTE remains limited, as single‑pass generation forces the model to determine span boundaries, opinion grouping, and sentiment polarity in a single decoding step. Common remedies, such as few‑shot in‑context learning and chain‑of‑thought prompting, offer only marginal improvements and rely heavily on either in‑domain demonstrations sampled from labeled training data or carefully engineered reasoning prompts, neither of which is broadly available in zero‑shot deployment. Inspired by the classical agent paradigm, we propose MASTE, a multi‑agent pipeline for zero‑shot Aspect Sentiment Triplet Extraction. MASTE decomposes ASTE into four sequential stages, where specialized agents handle different compositional subtasks with explicit conditioning on prior outputs. This design enables entirely training‑free zero‑shot ASTE and generalizes across different backbones and datasets. Extensive experiments on four ASTE benchmarks show that MASTE substantially outperforms zero‑shot and chain‑of‑thought LLM baselines under the same backbone, narrowing the gap to fully supervised methods without using any labeled triplets. Code is available at https://github.com/Hankerlove/MASTE.
Authors:Hang Fan, Weican Liu, Ying Lu, Dunnan Liu, Long Cheng, Wei Wei
Abstract:
Accurate photovoltaic (PV) power forecasting is essential for reliable grid dispatch and renewable energy integration, yet it remains challenging because PV generation is jointly shaped by weather variability, day‑night transitions, regime‑dependent dynamics, and strict physical constraints. We propose PARA‑PV, a Physics‑Aware Retrieval‑Augmented framework that embeds physical knowledge throughout the forecasting process. The framework first encodes multivariate PV observations into patch‑level representations and, through a physics‑aware retrieval‑augmented learner, retrieves historical patches and analog trajectories that are consistent with the current window in temporal shape, power level, PV operating state, and intra‑day period; this yields a physically grounded base forecast. To supplement local memory with broader temporal knowledge, the base forecast is then calibrated against a frozen Chronos time‑series foundation‑model prior through a lightweight residual adapter, so that general temporal regularities are adapted to PV‑specific dynamics without overriding the physically grounded prediction. Because residual conditional distribution shifts persist when weather and diurnal regimes change, a physics‑aware distribution shift correction module subsequently adjusts the preliminary forecast using power, weather, timestamp, and day/night conditions, applying gated mean‑shift and scale corrections selectively. Finally, a physics‑constrained loss function partitions the samples into peak, ramping, night‑time, and regular regimes and adaptively reweights their error contributions, preventing the dominant regular regime from suppressing learning of operationally critical states. Our code is available at https://github.com/weican1103/PARA‑PV.
Authors:Ruining Yang, Muxing Wang, Yixiao Chen, Tongfei Guo, Yi Xu, Can Cui, Zichong Yang, Yitian Zhang, Ziran Wang, Yun Fu, Lili Su
Abstract:
End‑to‑end models that map multimodal inputs directly to future trajectories/maneuvers have emerged as an increasingly prominent research paradigm in autonomous driving. This class of models includes both Vision‑Language‑Action models and trajectory‑generative planners. Unlike classic machine learning applications, autonomous vehicles operate in safety‑critical and interaction‑intensive environments where traditional open‑loop imitation of expert demonstrations is not sufficient to ensure reliability. In particular, small execution errors can accumulate over time, while recovery behaviors are scarce in training data. In addition, long‑horizon objectives such as safety and driving comfort are not captured by pointwise labels either. These limitations have motivated a shift toward post‑training techniques, which further refine driving policies beyond pure imitation. This survey presents a unified view of post‑training for autonomous driving by defining its scope and organizing the existing literature into four major families based on the form of supervision they use. For each family, we discuss its capabilities, limitations, and open challenges. We aim to facilitate a systematic understanding of this emerging area and stimulate future research on reliable and efficient post‑training for autonomous driving.A collection of related papers is available at https://github.com/RYNing/Awesome‑Post‑Training‑In‑Autonomous‑Driving‑Papers.
Authors:Xiucheng Wang, Junxi Huang, Nan Cheng
Abstract:
Angular radio maps describe the received‑power distribution over the angle of arrival and underpin beam selection and receiver localization in sixth‑generation (6G) networks. Predicting the angular power spectrum (APS) from geometry is difficult, because the mapping is ill‑posed in non‑line‑of‑sight (NLOS) conditions and must generalize to unseen environments. Distortion‑minimizing regressors return the conditional mean, which over‑smooths the spectrum and erases the multipath structure that downstream tasks need. We cast the task as a perception‑distortion problem and propose RadioDiff‑v2, a dual‑branch one‑dimensional diffusion transformer trained with flow matching. It couples periodic angular encoding, adaptive layer‑normalization conditioning, a Fourier angular mixer, and joint velocity and clean‑signal heads. A per‑metric estimator portfolio reads every deployment quantity from this single model, so that samples carry the distribution, the clean‑signal head supplies a regression‑grade point estimate, Bayes‑optimal rules select beams, and the conditional likelihood localizes the receiver. We prove that a concentrated conditional yields a straight probability‑flow trajectory that one step integrates exactly, identifying deterministic transport as the correct inductive bias. On a zero‑shot test of 99 environments and one million links, RadioDiff‑v2 leads every baseline on every metric, with a 0.39 dB Wasserstein‑1 distance, per‑bin error below the regression baseline, a 2.43 dB eight‑beam NLOS sweep loss, and a 20.6‑pixel localization error with four base stations. Code is available at https://github.com/UNIC‑Lab/RadioDiff‑v2.
Authors:Joongho Ahn, Moonsoo Kim
Abstract:
Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness‑engineering approach that reconstructs this pattern into a traceable, auditable LLM‑agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source‑backed claims remain the authority for runtime answers. We instantiate it on a public‑data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source‑grounding, entity‑routing, trace, output‑hygiene, and recommendation‑language contracts across the fixed validation scenarios; a fault‑injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition‑boundary runs; failures were confined to the model‑composed side and were caught and recorded. (3) The code‑owned guarantees are load‑bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation‑language and internal‑trace‑leakage violations reach the reader, which the harness blocks entirely. A bolt‑on external guardrail prevents such violations too but over‑refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code‑owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.
Authors:Emily Jin, Joy Hsu, Yiqing Xu, Weiyu Liu, Nick Haber, Jiajun Wu
Abstract:
Long‑horizon robot planning requires jointly reasoning over semantic task structure and geometric feasibility. To successfully execute a task, a robot must decompose goals, select task‑relevant objects, and sequence actions, while ensuring that plans satisfy spatial constraints such as limited free space and object collisions. In this work, we propose APIVOT, a VLM‑based planner that adaptively interleaves language and visual thoughts for long‑horizon planning. APIVOT learns to leverage language for semantic reasoning, while using visual thoughts as imagined future states for internal verification of geometric feasibility. On long‑horizon kitchen tasks, APIVOT outperforms general‑purpose VLMs and prior planning frameworks, achieving the largest gains in spatially constrained settings. We find that APIVOT learns meaningful modality selection behavior, demonstrating that adaptive interleaving of vision‑language thoughts improves both planning success and reasoning efficiency.
Authors:Shuo Huai, Di Liu, Hao Kong, Xiangzhong Luo, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin
Abstract:
Federated Learning (FL) empowers multiple clients to collaboratively learn a model, enlarging the training data of each client for high accuracy while protecting data privacy. However, when deploying FL in real‑time edge systems, the heterogeneity of devices among systems has a severe impact on the performance of the inferred model. Existing optimizations on FL focus on improving the training efficiency but fail to speed up inference, especially when there is a latency constraint. In this work, we propose Collate, a novel training framework that collaboratively learns heterogeneous models to meet the latency constraints of multiple edge systems simultaneously. We design a dynamic zeroizing‑recovering method to adjust each local model architecture for high accuracy under its latency constraint. A proto‑corrected federated aggregation scheme is also introduced to aggregate all heterogeneous local models, satisfying the latency constraint of different systems with only one training process and maintaining high accuracy. Extensive experiments indicate that, compared to state‑of‑the‑art methods and under a latency constraint, our extended models can improve the accuracy by 1.96% on average, and our shrunk models can also obtain a 3.09% accuracy improvement on average, with almost no extra training overhead. The related codes and data will be available at https://github.com/ntuliuteam/Collate
Authors:Seokhoon Jeong, Mijung Kim, Taehwan Kim
Abstract:
Neural architecture search (NAS) methods have grown increasingly efficient, yet they remain bounded by manually engineered search spaces that require substantial domain expertise and must be rebuilt for every new task. Large language models (LLMs) can generate architectures in an open‑ended space, but how to optimally divide the labor between LLM‑driven design and NAS‑driven search remains unexplored. We propose a mechanism that bridges these two paradigms: an LLM produces a high‑quality seed architecture, then decomposes it into a "slotted architecture", a scaffold with named, interchangeable module slots that automatically defines a bounded, task‑specific search space for conventional NAS to explore, without manual engineering. We instantiate this mechanism in AgentNAS, a modular three‑phase pipeline in which each component's contribution can be measured independently. On 17 tasks spanning classification, dense regression, segmentation, and multi‑label tagging across diverse modalities (NAS‑Bench‑360 and Unseen NAS), AgentNAS establishes a new state of the art on 11 tasks, outperforming published baselines including task‑specific expert designs. Ablation studies show that the two search mechanisms are broadly complementary: the LLM‑generated seed already surpasses published baselines on the majority of tasks, and NAS delivers additional gains in most cases through combinatorial recombination across slots, a mode of search that independent LLM samples cannot replicate. These patterns hold across three LLMs of different capability levels, confirming that the division of labor is robust. Our code is available at https://github.com/alroimfebruary/AgentNAS.
Authors:Xiuyi Lou, Zicheng Xu, Yu-Neng Chuang, Hoang Anh Duy Le, Zhaozhuo Xu, Guanchu Wang, Vladimir Braverman
Abstract:
Reinforcement learning (RL) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs). However, widely used critic‑free RL methods rely on uniform credit assignment, broadcasting the same advantage to all tokens regardless of their differences. We identify a critical failure mode of this design, which we refer to as Positive‑Credit Contamination: low‑probability tail tokens that are contextually erroneous receive identical positive credit to plausible ones within the same trajectory, resulting in the indiscriminate reinforcement of flawed reasoning behavior. To mitigate this issue, we propose Tail‑Aware Credit calibratiOn (TACO), a method that calibrates uniform credit assignment to suppress undesirable positive updates. TACO first computes a tail‑risk score that incorporates the local generation context to assess each token's risk of falling into the unreliable tail, distinguishing unexpected rarity from uncertainty‑driven exploration. TACO then uses this score to tune positive credit for risky tokens without removing their gradients entirely, so that recurring useful rare patterns can accumulate reinforcement while incidental noise is progressively dampened. Experimental results across three LLMs and eight benchmarks show that TACO consistently outperforms GRPO‑style baselines. Notably, TACO improves training stability, supporting sustained performance gains in long‑horizon RL. The source code is available at: https://github.com/xiuyilou/TACO.
Authors:Tommaso Cerruti, Tim Rieder, George Rowlands, Lingfeng Jin, Imanol Schlag
Abstract:
Self‑attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at long context. This paper presents a comparative study of softmax attention and four recent recurrent linear‑attention architectures: DeltaNet, Gated DeltaNet, Kimi Delta Attention, and Gated DeltaNet‑2. We express these mechanisms in a common recurrent‑memory notation, making explicit how they differ in expressivity, memory decay, erase and write control, training throughput, and implementation complexity. Our experiments center on 350M‑parameter models trained for 15B tokens, and include optimizer and learning‑rate comparisons, hybrid‑versus‑pure stack comparisons, sequence‑length runtime measurements, larger DeltaNet runs at 1.3B and 3B parameters, and a small set of downstream evaluations. The reported speed results measure training throughput and iteration time; we do not provide an empirical inference‑speed benchmark. Within the reported 350M‑parameter, 15B‑token sweep, Kimi Delta Attention with Muon reaches the lowest final validation loss, a pure Gated DeltaNet stack trained with AdamW has the highest normalized training throughput, hybrid stacks generally improve loss at a throughput cost, and Muon consistently lowers final validation loss relative to AdamW in the matched architecture settings we evaluate. We introduce and evaluate lightweight cross‑layer routing mechanisms for DeltaNet‑style memories. The most natural DeltaNet‑inspired formulation, forwarding a lower layer's delta‑rule write error into the next layer's value target, does not improve over matched baselines. Routing into the aligned hidden stream and forwarding the write value instead yields a modest improvement in the matched runs we report: Cross‑Layer Value Routing (CLVR) lowers final validation loss for both DeltaNet and Gated DeltaNet.
Authors:Wenqi Huang, Charley Lee, Leonard Tng, Serena Ge
Abstract:
DeepSWE is a benchmark of 113 original, long‑horizon software engineering tasks for evaluating coding agents. Most public agentic coding benchmarks follow SWE‑bench in mining merged fixes from public GitHub repositories, which creates two problems: the fixes and their discussion were likely seen during pretraining, so a high score can reflect recall rather than problem‑solving; and each task is graded by the tests that shipped with its merged fix, which were written to confirm one specific fix rather than grade an arbitrary solution, so they can fail a correct alternative or pass an incomplete one. DeepSWE avoids both. Its tasks are written from scratch across 91 active open‑source repositories and five languages and are never contributed back upstream, so their reference solutions stay out of the public record that model training scrapes; and each task is graded by a hand‑written verifier that checks the requested functionality and accepts any implementation that provides it. When an independent LLM judge re‑reviews graded runs, it disagrees with DeepSWE's verifier about an order of magnitude less often than with SWE‑Bench Pro's inherited tests (1.4% versus 32.4%). Despite being about half the length of SWE‑Bench Pro's prompts, DeepSWE's prompts describe tasks whose reference solutions touch 5.5x more code, and the benchmark separates frontier agents across a wider score band than the leaderboards on which they otherwise cluster. We release the benchmark, its verifiers, and the full record of evaluation trajectories.
Authors:Claudio Meggio, Johan Pensar, Riccardo De Bin
Abstract:
We present path_boost, a Python package for interpretable supervised learning on graph‑structured input data. The package implements PathBoost, a gradient boosting algorithm that automatically discovers predictive labeled paths within graphs during the learning process. Unlike graph neural networks, which are generally difficult to interpret, PathBoost produces an additive prediction model over path‑based features that explicitly reveals which substructures drive predictions. To avoid an exhaustive enumeration of all possible paths, the algorithm iteratively selects and extends paths during learning based on their predictive power, using boosting to combine weak learners into a strong ensemble. The package supports both regression and binary classification. Key features include compatibility with scikit‑learn workflows, support for custom base learners and selectors, automatic starting node selection, parallel training across anchor nodes, and built‑in variable importance computation. We demonstrate PathBoost on molecular property prediction of transition metal compounds, where atoms serve as nodes and bonds as edges, and further benchmark PathBoost against an established graph neural network and a graph kernel method across six molecular datasets. The package is available on PyPI and GitHub under an open‑source license.
Authors:Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu, Vaidehi Patil
Abstract:
With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross‑modal associations that originate from their training data. Retraining after deletion requests or policy updates is often impractical, and targeted forgetting remains difficult because knowledge is distributed across shared representations. Multimodal unlearning addresses this challenge by enabling selective removal across modalities while retaining overall utility. This survey offers a unified, system‑oriented view of multimodal unlearning across vision, language, audio, and video, grounded in recent advances, emerging applications, and open problems. Our taxonomy enables systematic comparison across model architectures and modalities, clarifying trade‑offs among deletion strength, retention, efficiency, reversibility, and robustness. This survey highlights open problems and practical considerations to support future research and deployment of multimodal unlearning. We release a curated repository: https://smsnobin77.github.io/Awesome‑Multimodal‑Unlearning/
Authors:Zachary Charlick, Nilay Roy Choudhury, Haoyu Ma, Xiaonan Huang, Dmitry Berenson
Abstract:
The scalability of organic agriculture is partially limited by the labor costs associated with monitoring for pests. While drones and rovers are well‑suited for agricultural monitoring from above or next to plants, many pests live on the underside of leaves or on plant stems, making them detectable only after they have caused significant damage. To enable early pest detection we present STEMbot, a miniature climbing robot system designed for autonomous navigation under plant canopies. Unlike existing climbing platforms that lack on‑board perception or are restricted to unbranched vertical trunks, STEMbot integrates a fully geometric PIN‑SLAM pipeline with a semantic OcTree to achieve robust localization and mapping while climbing the plant. To plan STEMbot's motion we propose a manifold‑constrained A planner along with ray‑tracing goal specification to enable branch‑aware traversal and the inspection of occluded targets. We validate our system through hardware experiments, demonstrating reliable traversal of stems ranging from 7‑33mm and autonomous navigation across four distinct plant specimens. Quantitative evaluations show that our system achieves high‑fidelity geometric reconstructions with an average Chamfer distance of less than 1cm relative to an offline photogrammetry baseline, confirming that STEMbot maintains the globally consistent odometry needed for autonomous navigation.
Authors:Sirui Lu, Erickson Tjoa, J. Ignacio Cirac
Abstract:
We build a team of specialized large language‑model agents and present an agent‑driven workflow for research‑level formalization in theoretical physics, with the autoformalization of the fundamental theorem of matrix‑product states as a demonstration. The agents, coordinated through a structured mathematical blueprint and periodic human review, orchestrated and executed the full formalization autonomously. For some statements, the agents were able to explore new proof routes that are not part of the standard literature. Along the way the agents produced extensive tensor‑network and quantum‑information libraries not previously available in Mathlib, Lean's mathematical library. As a physical application, the formalization also extends towards symmetry‑protected topological phases in one dimension. We find that the main bottleneck in large‑scale autoformalization is enforcing mathematical intent and we provide a detailed study of the full process and various subtleties involved. We release the codebase as the library \hrefhttps://github.com/LionSR/TNLeanTNLean, together with a \nChapters‑chapter \hrefhttps://lionsr.github.io/TNLean/blueprint/blueprint of the formalization effort.
Authors:Erdemt Bao, Xing Lei, Jun Chen
Abstract:
Hierarchical Implicit Q‑Learning (HIQL), an offline goal‑conditioned RL method, selects subgoals by value‑function advantages alone. This rule has two coupled failure modes. Optimistic bias treats lucky stochastic outcomes as skillful choices, and mode collapse reduces a multi‑modal subgoal distribution to a single Gaussian mean that often falls in unreachable regions. We propose NFTR (Normalizing Flows subgoal policies with Triangle‑slack Reweighting). A conditional Normalizing Flow replaces the Gaussian policy, and a closed‑form mode‑averaging result identifies NFs as the minimal generative class for AWR‑based subgoal selection. A triangle slack score, built on the architectural triangle inequality without relying on distance accuracy, multiplicatively corrects the AWR weight to downweight subgoals whose detour cost exceeds average reachability. Triangle‑slack vanishes on geodesics in deterministic MDPs and remains a conservative upper bound on composability violation under stochastic dynamics. The RWDR objective preserves AWR's population‑level monotonic improvement and admits a three‑term suboptimality decomposition. Together, these two ingredients yield subgoal selection that provably avoids the Gaussian collapse described above and remains stable under stochastic dynamics. GitHub page: https://github.com/erdemtbao/NFTR
Authors:Anne Harrington, Nayan Saxena, Michael Murphy, Anastasia Borovykh, Zeyu Yun, Sridhar Kamath, Ara Eindra Kyi, Trevor Darrell, Jitendra Malik, Yutong Bai
Abstract:
As large language models (LLMs) become increasingly capable, the next question is how can we enable models to continually learn? Today, the field largely frames this as a problem of context management and mitigating forgetting. We argue this framing is incomplete: continual learning is fundamentally about increasing model competence as the world changes. We disentangle this change along two axes ‑‑ space, where the model encounters new domains, and time, where the underlying data drifts under a fixed task. This framing lets us study continual learning under realistic conditions: new domains arrive over time, facts drift past their training cutoff, and agentic interactions accumulate state across episodes. To evaluate methods under this setting, we recast widely used LLM benchmarks as sequential problems and introduce a single mechanism‑agnostic protocol that compares prompt‑based methods (GEPA, ACE), supervised learning (SFT, SDFT), reinforcement learning (GRPO, SDPO), and context compression (Cartridges, In‑place TTT). Prompt‑based methods fit each new stage quickly but degrade on future tasks. Distillation‑based methods accumulate knowledge stably but struggle to update outdated facts. Context compression improves efficiency without substantially improving the ability to learn new tasks. Online reinforcement learning adapts most effectively to knowledge updates but remains sensitive to noisy reward signals. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding. We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems.
Authors:Julian Shen, Ludwig Schmid, Robert Wille
Abstract:
Silicon spin qubits have emerged as a promising qubit technology due to their favorable scaling and fabrication properties. However, efficiently compiling quantum circuits onto spin qubit platforms remains challenging, particularly when accounting for hardware constraints and the high sensitivity to static defects. Existing compilation approaches for spin qubits either largely ignore error correction, despite its critical role for large‑scale quantum computation, or focus on low‑level schedule constructions, missing a high‑level compilation and routing for logical, error‑corrected algorithms. To address this gap, we introduce a compilation framework for spin qubits based on the recent snakes on a plane model, which utilizes a 2D surface code and qubit teleportation to mitigate errors. Building on this model, we propose shortest‑path and rotation‑based algorithms as two novel classes of qubit‑routing techniques, along with additional defect‑handling and initial‑mapping strategies. We evaluate both algorithms across diverse architectural settings and problem sizes, demonstrating that shortest‑path methods excel in sparse, low‑defect scenarios, while rotation‑based approaches perform better in high‑density environments. An open‑source implementation of our framework is publicly available on GitHub as part of the Munich Quantum Toolkit (MQT) at https://github.com/munich‑quantum‑toolkit/spin‑qubit‑routing.
Authors:Michal Widera
Abstract:
We present RetractorDB, an open‑source edge signal processing engine (ESPE) for regular time series whose query semantics is grounded in the number theory of covering systems. RetractorDB is designed to support, not replace, time‑series databases (TSDB) and data stream management systems (DSMS): deployed close to the signal source, it pre‑processes and filters high‑frequency measurements on the edge device through a declarative signal‑processing query language, maintains a partial, correctable record of past and scheduled future events in inspectable artifacts, and transmits exact, deterministic results upstream, so that only reduced, already‑processed streams reach the central architecture. The data model is differential (a stream is a pair (s_n, Δ) with a constant rational inter‑arrival interval), and the core rate‑conversion operators, interleave and de‑interleave, are proved to be rational Beatty sequences satisfying the conditions of Fraenkel's partition theorem. This yields an algebra in which resampling is an exact, deterministic, first‑class operator: de‑interleaving inverts interleaving bit‑for‑bit using rational arithmetic alone, and algebraic rewrite rules license query‑plan optimization without changing results. We describe the end‑to‑end realization of this algebra in a working engine: declarative query language (RQL), compilation to a dependency DAG with rational interval resolution, slot‑based runtime scheduling, and an inspectable artifact format with schema and null/gap metadata. We validate the semantics on deterministic query examples drawn from the engine's integration tests, including a complete Pan‑Tompkins QRS‑detection pipeline over MIT‑BIH ECG data expressed entirely within the algebra. A performance evaluation under a real‑time operating environment is in progress and deferred to a subsequent version.
Authors:Zhoujie Hou, Song Wang, Kexin Lou, Mo Wang, Chen Wei, Quanying Liu
Abstract:
Sleep physiology arises from the coordinated dynamics of the central nervous system (CNS) and autonomic nervous system (ANS), as reflected by multimodal polysomnography signals including EEG, EOG, EMG, ECG, and respiration. However, existing sleep foundation models often fuse heterogeneous biosignals in a topology‑agnostic manner, overlooking their physiological organization. We introduce Omni‑Sleep, a sleep foundation model that uses the CNS/ANS partition as a physiological prior for topology‑constrained representation learning. Omni‑Sleep learns structured representations through three objectives: intra‑system consistency, which captures shared subsystem‑level factors within neural and cardio‑respiratory signals; inter‑system synchronization, which aligns subsystem trajectories to model brain‑‑body dynamics; and latent‑space masked temporal modeling, which captures long‑horizon sleep dynamics. Pre‑trained on over 100,000 hours of multi‑center multimodal PSG data, Omni‑Sleep is evaluated on sleep staging and multi‑disease classification. Across datasets and modality‑ablation settings, Omni‑Sleep outperforms strong foundation‑model baselines, showing improved label efficiency, cross‑dataset generalization, and robustness to missing modalities. These results highlight the value of physiological hierarchy for generalizable sleep representation learning. Code is available at https://github.com/AutoBrain‑sleep/OmniSleep.
Authors:Wentao Lu
Abstract:
Parameter‑efficient fine‑tuning adapts a large language model to one task cheaply, but across a task sequence LoRA‑style methods keep stacking low‑rank updates on the same frozen weight, so each new task tends to overwrite the previous ones. We present ReCoLoRA (Recursive Consolidation of Low‑Rank Adapters), a spectrum‑aware framework for continual fine‑tuning: adapters are initialized from a randomized SVD of the pretrained weight, per‑layer effective ranks are selected by an elbow criterion, and the principal subspace is adapted before residual capacity is opened. Before each new task, ReCoLoRA re‑decomposes the current effective weight, rather than the original one, into a frozen residual, a slowly updated principal component, and a fresh adapter (recursive consolidation), so every task starts from the model that has already absorbed its predecessors. On a six‑task continual GLUE sequence over four 7‑8B backbones, ReCoLoRA attains the best final average score on three of the four backbones against rank‑swept LoRA, PiSSA, AdaLoRA, and DoRA baselines while training fewer parameters; an oracle‑routed task‑bank variant serves as an upper bound under full task isolation. Code: https://github.com/bhqy666/ReCoLoRA.
Authors:Yazheng Liu, Xi Zhang, Sihong Xie, Hui Xiong
Abstract:
Temporal graphs are ubiquitous in real‑world applications and Temporal Graph Networks (TGNs) have achieved superior predictive accuracy. Understanding which historical events drive model predictions can enhance trustworthiness of TGNs. Existing explanation methods overlook the memory module, the core component that records and updates node histories, leaving the influence of past events unexplored. To address this, we attribute TGNs predictions through the topology attribution tree and memory backtracking tree. The topology attribution tree captures the influence of neighbors and their memory vectors, then the memory backtracking tree quantifies how historical events shape node memory vectors. We apply the LRP in TGNs, ensuring that the total contribution of events equals the logits of model. Finally, top‑k selection may be unfaithful due to the nonlinear mapping from logits to probabilities, we design optimization objectives to identify the important events. Experiments on nine temporal graph datasets, spanning node property prediction, link prediction tasks and graph classification tasks, show that our method provides faithful explanations and outperforms state‑of‑the‑art baselines. The code is available at https://github.com/yazhengliu/MemExplainer
Authors:Ying Chang, Jiahang Xu, Xuan Feng, Chenyuan Yang, Peng Cheng, Yuqing Yang
Abstract:
The optimization of long‑horizon agents increasingly relies on reflection‑based mechanisms, where a large language model (LLM) acts as an optimizer to diagnose agent failures and improve agent policies. However, real execution traces are difficult to use directly for optimization: large trace collections are often redundant and heterogeneous, making optimization inefficient and prone to overfitting to low‑value failures; meanwhile, each individual trajectory also contains many irrelevant steps, while naive context reduction methods such as truncation or sliding windows can discard causally important evidence and produce misleading optimization signals. To resolve this dilemma, we introduce STRACE (Structural TRajectory Analysis and Causal Extraction), a framework that constructs high signal‑noise optimization contexts for more precise and effective optimization. At the batch level, STRACE mines failure patterns to filter redundant traces and retain representative failures; within each selected trace, it performs causal localization over a textual dependency graph to remove non‑causal steps and identify the true root‑cause module for optimization. Empirical results demonstrate that STRACE significantly outperforms standard context‑filtering baselines. Notably, on a challenging formal verification task (VeruSAGE‑Bench), it successfully optimizes human‑expert designed agents, delivering 1.4× success‑rate improvement (42.5% to 58.5%). The code is available at https://github.com/moomight/STRACE .
Authors:Tianming Sha, Yue Zhao, Lichao Sun, Yushun Dong
Abstract:
Autonomous AI agents can execute complex tasks with limited human review, yet they often lack the grounded operational knowledge to make their outputs not just executable but correct, secure, and maintainable. We introduce SkillCenter, to our knowledge the largest open skill library for agents by total count: 216,938 structured skills across 24 domain bundles. A SkillGate‑filtered pipeline contributes 114,565 source‑grounded skills from peer‑reviewed journals, ArXiv, and over 24,000 technical sources, integrated with 102,373 community skills from GitHub and the ClawHub marketplace. We present the end‑to‑end framework that builds the pipeline subset: multi‑source acquisition, an LLM‑based quality gate (SkillGate), template‑driven generation, iterative source‑grounding, and quality‑controlled publishing. Source grounding is a traceability guarantee: each retained claim maps to an exact quotation in its source. All skills ship as offline‑searchable SQLite FTS5 bundles.
Authors:Hongyu Qu, Jianzhe Gao, Xiaobin Hu, Shaohuan Yang, Xinlei Yu, Rui Yan, Wenguan Wang, Xiangbo Shu, Shuicheng Yan
Abstract:
Mainstream Vision‑Language‑Action (VLA) models predict actions primarily from the current observation under a Markovian assumption, thus struggling with long‑horizon, temporally dependent tasks. Existing memory‑augmented VLAs either expand the observation window or retrieve history from the memory bank as auxiliary policy‑side context. However, they leave memory outside the native latent embedding space of VLA reasoning, preventing historical experience from being fluidly interleaved with multimodal reasoning and action formation. To this end, we introduce LaMem‑VLA, a latent‑memory‑native framework that reconstructs historical experience into latent memory tokens and directly interweaves them with VLA reasoning. At its core, LaMem‑VLA introduces four coordinated components: (i) a curator that organizes historical experience into two complementary short‑term and long‑term memory vaults; (ii) a seeker that queries both vaults using the multimodal cognition to retrieve context‑relevant evidence; (iii) a condenser that reconstructs the retrieved evidence into compact short‑term and long‑term latent memory tokens; and (iv) a weaver that injects these memory tokens with the current observation and instruction into one continuous embedding sequence. By representing, retrieving, and consuming historical experience entirely in the same continuous latent space, LaMem‑VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context. Extensive experiments on SimplerEnv and LIBERO demonstrate the superiority of our LaMem‑VLA.
Authors:Maximilian Andreas Hoefler, Karsten Mueller, Wojciech Samek
Abstract:
One‑shot federated learning (OSFL) addresses the communication overhead of federated learning by limiting training to a single round, but doing so without sacrificing model quality is non‑trivial, particularly when client data distributions diverge. Recent work has addressed this challenge by aggregating client knowledge on the server through the construction of transferable synthetic datasets or distillates. However, most of these methods lack formal privacy guarantees, leaving a gap in jointly achieving low communication, robustness to heterogeneity, and rigorous privacy. We propose FedKT‑CSD (Federated Knowledge Transfer via Collaborative Synthetic Data), a framework inspired by neural image compression that closes this gap by leveraging publicly pretrained autoencoders as a shared latent space. Each client encodes its private data in a single forward pass, computes class‑conditional latent statistics, and transmits these to the server. The server aggregates these statistics via secure aggregation, adds calibrated differential privacy noise, and decodes a synthetic dataset for training a global model and further downstream tasks. This design provides formal (\varepsilon,δ)‑differential privacy by construction, while keeping client‑side computation and communication lightweight. Despite operating under privacy constraints, FedKT‑CSD is competitive with and even outperforms non‑private baselines across diverse datasets and heterogeneity settings, and scales to a large number of clients. Our code is available at: https://github.com/an7123/FedKT‑CSD
Authors:Qinnan Cai, Yibo Zhao, Xiang Li
Abstract:
Large language model based search agents increasingly adopt multi‑agent architectures in which a main agent decomposes a complex question into sub‑queries and dispatches them to parallel sub‑agents. However, existing systems instantiate all roles from a single model of identical scale, leaving open how model capacity should be distributed across roles. We factorize hierarchical search into three roles: a delegation role responsible for task decomposition, an execution role responsible for retrieval and evidence extraction, and an answer generation role held fixed as a confound control. We then conduct controlled capacity sweeps along the delegation and execution axes on five multi‑hop QA benchmarks. The experiments yield three findings. First, role factorization consistently outperforms a single‑agent baseline, improving exact match from 4.5 to 8.6 points across six model scales. Second, capacity sensitivity is asymmetric: scaling the delegation backbone improves EM by ~11 points, whereas scaling the execution sub‑agent moves EM by only ~2.6 points, identifying decomposition as the capability bottleneck. Third, a 1.7B‑parameter executor trained via quality‑filtered trajectory distillation matches a frontier sub‑agent in accuracy while consuming 37% fewer sub‑agent tokens, advancing the Pareto frontier. These results suggest a concrete recipe for building hierarchical search agents: concentrate capacity at delegation and downsize execution without sacrificing accuracy. Our code is available at https://github.com/QinnanCai0115/role‑factorized‑search.
Authors:Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, Jiahao Wang, Yufeng Yuan, Hanlin Wang, Yichong Lu, Ka Leong Cheng, Haojie Zhang, Jian Gao, Tianrui Feng, Yuzheng Liu, Yao Yao, Yinghao Xu, Xing Zhu, Yujun Shen, Hao Ouyang
Abstract:
We present LingBot‑World 2.0 (also known as LingBot‑World‑Infinity), an advanced iteration of LingBot‑World featuring four distinct upgrades. (1) Our model achieves an unbounded interaction horizon while maintaining consistent output quality, benefiting from a carefully crafted causal pretraining paradigm. (2) Through distilling a real‑time variant from the base model, our system guarantees rapid response time, sufficient to drive 720p video streams at 60 fps. (3) Compared to the previous version, this update introduces highly diverse interactive elements, comprising a broader spectrum of actions (e.g., attacking, archery, spell‑casting, and shooting) alongside a richer variety of text‑driven events. (4) We pioneer the integration of an agentic harness within the domain of world modeling, wherein a pilot agent is tasked with planning and executing character behaviors, while a director agent is responsible for synthesizing novel environmental elements as the scene progresses. Additionally, to facilitate a shared experience, we develop an interface that permits multiple players to simultaneously immerse themselves in this vivid world simulator. We pair our primary 14B model with a lightweight 1.3B counterpart, which supports effortless deployment on a single GPU.
Authors:Feng He, Zhenting Wang, Qifan Wang, Qiang Guan, Dongfang Liu, Ruixiang Tang, Qiankun Li
Abstract:
Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence. Prior work mainly focuses on detecting or suppressing hallucinations at generation time, leaving the subsequent reasoning stage largely unexplored. In this work, we study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model's inference context and influence downstream predictions. To systematically investigate PHR, we introduce HIVE, Hallucination Inference and Verification Engine, an evaluation infrastructure that enables controlled comparisons between faithful and hallucinated captions. Across nine tasks and nine models, we observe structured modality dependent patterns: hallucinated captions often improve accuracy on vision language tasks, while text only tasks exhibit limited or unstable effects. Further analyses show that hallucinated cues broaden semantic coverage and reshape reasoning dynamics while preserving stable inference. These findings highlight that hallucinated semantics may influence downstream reasoning once they enter the model's inference context. Understanding this post hallucination stage is important for improving the reliability and interpretability of multimodal reasoning systems. Code is publicly available at https://github.com/hefengcs/HIVE.
Authors:Jaris Küken, Shi Bin Hoo, Martin Mráz, Frank Hutter, Lennart Purucker
Abstract:
Time series classification (TSC) is dominated by a two‑stage paradigm: train a feature encoder ‑‑ either from scratch on the target dataset or via pretraining on large corpora ‑‑ and then fit a task‑specific classifier on top. While effective, this decoupling optimizes representation learning independently of the classification objective, requires per‑dataset training, and prevents the model from exploiting label information during inference. We introduce TimEE, a 4.5M‑parameter foundation model for end‑to‑end TSC via in‑context learning. Given a labeled support set and a query time series, TimEE directly outputs a predicted class distribution in a single forward pass with no per‑dataset training required. Following the prior‑data fitted network (PFN) framework, TimEE is meta‑trained exclusively on synthetic TSC tasks, where each task contains time series with distinct class identities arising from structured distributional shifts in the generative process. Despite seeing no real time series during pre‑training, TimEE ranks first in ROC AUC (and third on accuracy) on the UCR benchmark among all compared methods, which include both foundation models and supervised deep learning baselines. To our knowledge, TimEE is the first purely synthetic‑pretrained model to reach state‑of‑the‑art performance on the UCR benchmark. These results establish end‑to‑end ICL with synthetic priors as a compelling, largely unexplored direction for TSC, with scaling, prior design, and richer generation mechanisms as natural avenues for improvement. Code is publicly available at http://github.com/automl/timee.
Authors:Harry Owiredu-Ashley
Abstract:
Agentic red‑teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that this binary attack‑success rate discards the information a defender most needs, namely how harmful the resulting action was. We introduce an action‑graded harm rubric that scores an agent's tool‑call trajectory on a seven‑level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege. We compute the scale two ways: a deterministic oracle that reads the trajectory and the attacker's stated goal, and a panel of three frontier language‑model judges that read a tag‑free account of the same trajectory. Across four victim models and two defenses on the AgentDojo workspace suite, severity grading exposes three cases the binary metric hides, including a defense that reports a zero attack‑success rate while still permitting an externally visible cross‑scope leak through an unfiltered tool. The judge panel reproduces the oracle with high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots that we characterize, most notably a failure to recognize escalation chains. Unlike prior work that provides harm taxonomies, harmful‑task completion tests, execution‑level safety benchmarks, or severity‑aware simulation, our contribution is a reusable, trace‑grounded severity instrument applied to the actual actions recorded in existing red‑team logs. All code, prompts, and per‑episode logs are released.
Authors:Vinícius Gabriel Angelozzi, Héber H. Arcolezi
Abstract:
Machine learning models are increasingly deployed in high‑stakes domains, raising concerns about both privacy and fairness. Differential Privacy (DP) has become a gold standard for privacy‑preserving data analysis, while fairness‑aware mechanisms aim to mitigate discrimination against underrepresented groups. However, these objectives can conflict: DP often amplifies disparities across demographic groups, and little is known about whether established fairness interventions remain effective under DP constraints. In this work, we present, to our knowledge, the first systematic evaluation of fairness interventions on differentially private synthetic tabular data. Our benchmark centers on the Adaptive Iterative Mechanism (AIM), identified as the state‑of‑the‑art marginal‑based DP synthesizer (Cormode et al. 2025). We thus evaluate fairness interventions across four datasets, multiple group fairness metrics, and three categories of mitigation strategies (pre‑processing, in‑processing, and post‑processing) under a wide range of privacy budgets. We compare four pipeline configurations: (Baseline) training on original data; (DP‑only) training on DP synthetic data; (Fair‑only) applying fairness mechanisms on original data; and (DP+Fair) combining fairness mechanisms with DP synthetic data. Our results demonstrate that while DP alone can degrade both utility and fairness, applying fairness interventions can partially restore equitable outcomes. Among them, post‑processing methods tend to provide more stable fairness‑utility trade‑offs across privacy budgets and synthesizers, achieving strong fairness improvements while preserving competitive utility relative to other intervention stages. We release all code, data, and experimental artifacts in an open‑source repository to ensure full reproducibility and to support future research on the privacy‑fairness‑utility trade‑off.
Authors:Songhan Wang, Haoang Chi, He Li, Zhiheng Zhang, Jiayan Yuan, Cheems Wang, Hao Peng, Xinwang Liu, Wenjing Yang
Abstract:
Spatial and Single‑cell transcriptomics are transformative in deciphering cellular dynamics. As the fundamental paradigm for reconstructing cell developmental paths, trajectory inference (TI) is critical. However, existing methods require extensive manual intervention and proficiency in heterogeneous tools, posing a significant barrier to efficient TI analysis. To bridge this gap, we propose SpaCellAgent, an autonomous large language model (LLM) multi‑agent framework that automates end‑to‑end spatiotemporal analysis and narrative generation. SpaCellAgent utilizes a multi‑agent architecture for strategic workflow planning, a dynamic tool‑orchestration engine for adaptive algorithm selection, and a self‑evolution module that iteratively refines performance through feedback. We evaluate SpaCellAgent on six heterogeneous datasets encompassing complex temporal developmental trajectories, diverse sequencing platforms, and spatially‑resolved tissue architectures. SpaCellAgent consistently demonstrates over 40% improvement in analytical efficiency while maintaining expert‑aligned performance. By converting natural language specifications into optimized analytical workflows and fully automating the pipeline, SpaCellAgent democratizes advanced spatiotemporal modeling and establishes a scalable, agent‑driven paradigm for computational biology. The code and materials are available at https://github.com/LittleXH‑shw/SpaCellAgent.
Authors:Mayank Kharbanda, Michael Cochez, Rajiv Ratn Shah, Raghava Mutharaju
Abstract:
Logical Multi‑Hop Query Answering over Knowledge Graphs (KGs) can be formulated as querying, with an implicit completeness assumption. Current works mainly focus on Existential First Order Logic (EFO) queries. These EFO queries contain conjunction, disjunction, and negation operators. Most existing works employ transductive reasoning, meaning they are not capable of reasoning over entities unseen during training. In the real world, there is a resource scarcity, and we cannot train a model with all the nodes of a large KG. Hence, we propose InductWave, a wavelet‑based inductive embedding method for logical query answering on large KGs. Here, the training graph consists of fewer nodes than the test graph. Our model performs on par with the baseline models while having half the number of message‑passing layers. It outperforms all of them in most cases, with 75% of the layers. These fewer resource requirements enable us to evaluate InductWave on massive graphs, such as Wiki‑KG. We test our model using extensive experiments across varying train‑test graph proportions of the FB15k‑(237) dataset, comparing it with the state‑of‑the‑art models. The code and datasets for the model are available at https://github.com/kracr/inductwave/.
Authors:Guillaume de Romémont
Abstract:
We present JAX‑FVM, an open‑source, fully differentiable finite volume method (FVM) for the two‑dimensional compressible Euler and Navier‑Stokes equations on unstructured triangular meshes. The solver is written entirely in JAX, so that every operation : mesh connectivity, flux evaluation, slope limiting, and time integration is just‑in‑time compiled, vectorised, and end‑to‑end differentiable through automatic differentiation (AD), and runs transparently on CPU or GPU. On the numerical side, JAX‑FVM is built around an entropy‑conservative Tadmor/Ismail‑Roe two‑point flux supplemented with entropy‑variable Rusanov or Roe dissipation, second‑order MUSCL reconstruction of primitive variables with least‑squares gradients and Venkatakrishnan limiting, and a family of explicit (RK2‑4) and matrix‑free implicit (Newton, SDIRK2) time integrators whose Jacobian actions are obtained by AD. The combination of an unstructured‑mesh compressible FVM with end‑to‑end differentiability fills a gap left by existing differentiable CFD frameworks, which are almost exclusively restricted to structured grids or spectral discretisations. We describe the governing equations, the discretisation, the software architecture, and a set of standard verification cases. The code is openly available at https://github.com/guigzair/jax_fvm.
Authors:Shrikant Tangade, Bansi Pambhar, Valeria Loscri, Mauro Conti
Abstract:
The automotive industry is transitioning to Zonal‑oriented Architectures (ZoA) for Software‑Defined Vehicles (SDVs), enabling frequent over‑the‑air (OTA) updates for 100+ Electronic Control Units (ECUs). While OTA updates improve efficiency, they introduce safety‑critical security risks. Current standards like Uptane and AUTOSAR Adaptive rely on Public‑Key Infrastructure (PKI). However, PKI‑based authentication creates bandwidth bottlenecks in in‑vehicle and vehicle‑to‑cloud (V2I) communication as ECU density increases. It also risks exposing sensitive vehicle configurations and passenger privacy due to centralized architectures. Next‑generation Zonal SDVs require decentralized, scalable authentication with data privacy. To address this, we propose zk‑ScalHard, a hardware‑rooted, privacy‑preserving authentication protocol. We introduce a decentralized, hierarchical trust‑promotion model utilizing Silicon Physical Unclonable Functions (PUFs) and two novel Zero‑Knowledge Proof (ZKP) circuits: (1) Zonal Identity and Integrity (ZIDI) and (2) High‑Performance Computing Aggregation (HPCA). These circuits employ multi‑party computation (MPC) and recursive aggregation to achieve decentralization and scalability. The integration of ZKPs and PUFs ensures 100% vehicle‑level data sovereignty. Benchmarked against Uptane, zk‑ScalHard achieves constant O(1) communication and verification complexity, improving upon the linear O(n) complexity of current systems. Evaluation shows a 99.2% reduction in authentication bandwidth and a 99.9% reduction in the temporal attack surface. Our results demonstrate that zk‑ScalHard provides a scalable, secure, and GDPR‑compliant architecture for future Zonal SDVs.
Authors:Aswin Chandrasekaran
Abstract:
Online truckload bid acceptance is a closed‑loop stochastic decision problem in which a carrier or broker must, in real time, accept or reject a tendered load subject to operational feasibility, fleet repositioning costs, and opportunity cost against future demand. Public, reproducible benchmarks for this problem are scarce: existing routing benchmarks are static, while dynamic‑fleet studies typically rely on private operator data. We introduce FreightBidBench, a public‑calibrated, dependency‑free, closed‑loop benchmark in which feasibility (pickup reach, appointment windows, simplified hours‑of‑service, stochastic yard delays) and economics (service‑failure penalty, terminal fleet value, daily price‑premium window) are explicit, versioned, and reproducible from public Freight Analysis Framework and U.S. Department of Agriculture truck rate data. We develop two full‑horizon hindsight ceilings: a simple LP style relaxation and a tighter Lagrangian‑per‑truck information relaxation that retains per‑truck hours‑of‑service and sequencing structure and is 20.7% tighter than the LP relaxation on a tight‑capacity scenario and 39.3% tighter on a scarce‑capacity scenario. We introduce a parametric surrogate‑rollout cascade with boundary‑band and scarcity‑pressure escalation triggers. On ten‑seed tight and scarce scenarios, the best simple policy retains 91.0% and 86.5% of rollout profit and the standard‑library surrogate 94.2% and 89.3%; a cascade at a single escalation band recovers roughly 98% on both at 40‑56% of rollout's mean decision latency, and on the tight scenario is statistically indistinguishable from the rollout teacher (paired‑bootstrap 95% CI on the profit delta spans zero).
Authors:Reem AlYabis, Fares AlTuwaim, AlJawharh AlOtaibi, Mohamed Eltahir
Abstract:
Automated crowd counting in Hajj video is difficult not because current models lack capacity, but because the footage violates the assumptions those models were built on: cameras observe the crowd from steep, near‑vertical angles, individuals occlude one another extensively, and a single frame can contain well over a thousand people. Benchmarks that test crowd counting in such an environment are either private or not detailed per second. We revisit the HAJJv2 dataset and contribute HAJJv2‑CrowdCount: per‑second human‑annotated crowd counts for its testing videos. Using these annotations, we benchmark three recent zero‑shot counting paradigms: an open‑vocabulary detector (YOLO‑World), a point‑based counter (APGCC), and a promptable segmentation‑based counter (SAM3Count). SAM3Count attains the lowest overall mean absolute error (MAE 70.4, 95% CI 56.0‑86.1), ahead of YOLO‑World (92.0) and APGCC (152.9). This ordering reverses, however, in the regime most relevant to deployment: on the densest frames, the detection‑ and segmentation‑based counters both degrade sharply (MAE exceeding 300), while the point‑based counter degrades far more gracefully (MAE 114.9). This inversion is decision‑relevant for Hajj crowd management, where reliable counts are needed most precisely in the densest and most occluded scenes. The annotations are released to support reproduction and extension of these results.
Authors:Ahsan Habib Akash, Dipkamal Bhusal, Stacey Jones, Donald A. Adjeroh, Binod Bhattarai, Prashnna Kumar Gyawali
Abstract:
Deep neural networks are widely deployed in high‑stakes visual applications where interpretability is critical, yet existing explanations face a trade‑off: post‑hoc concept methods recover factors that are faithful to a model's behavior but unnamed, while naming and by‑design methods attach human‑readable concepts only by retraining or altering the classifier. We propose Language‑Anchored Decomposition (LAD), a post‑hoc framework that delivers concepts which are simultaneously named, faithful, and obtained without modifying the model. For each class, a large language model proposes a concept vocabulary that CLIP‑based similarity maps localize across image regions. Inverting standard non‑negative matrix factorization, LAD fixes these language‑grounded maps as the coefficient matrix and learns only a concept basis that reconstructs the frozen encoder's activations, so naming becomes a structural constraint and the model's own feature geometry determines which concepts are retained. Removing this anchor preserves accuracy but collapses attribution faithfulness. Across natural‑image, scene, and medical‑imaging benchmarks, LAD produces spatially precise explanations that are decision‑relevant under both concept insertion and deletion, while uniquely providing stable, human‑interpretable concept names.
Authors:Wenqiang Xiao, Wenzhuo Ma, Junxi Zhang, Zhenzhong Chen
Abstract:
Perceptual quality enhancement of severely compressed videos remains challenging due to complex artifact patterns and substantial information loss. Recent diffusion models have demonstrated strong generative capability for visual restoration, but directly applying them to compressed video often ignores compression degradation characteristics and may introduce structure‑inconsistent hallucinations. To address this issue, this paper presents a diffusion‑based compressed video enhancement method, named DiffCVE. Coding Prior‑enhanced Dual Conditioning (CPDC) branches are designed to jointly model compressed video and coding prior conditions, where coding priors including residuals and motion vectors provide complementary structural and motion guidance during the diffusion denoising process. To make the diffusion process aware of compression severity, a Compression Degradation Semantic Prompting (CDSP) mechanism is introduced to leverage QP‑conditioned textual prompts together with LoRA fine‑tuning. In addition, a Coding Prior‑guided Weighted Fusion (CPWF) module is incorporated into the VAE decoder to fuse VAE encoder and coding prior encoder features with QP‑predicted weights. Extensive experiments demonstrate the effectiveness of the proposed method in improving perceptual quality, especially under severe compression settings. The project page with enhanced video demonstrations is available at https://wqmaker.github.io/projects/DiffCVE/.
Authors:Ethan Chung, Chuanjun Zheng, Jasper Tan, Jingxi Li, Haopeng Zhang, Huaijin Chen
Abstract:
Vision‑language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the physics and inverse problems that underlie computational imaging. We present ImagingBench, a benchmark of 20 computational imaging tasks spanning five categories: ray and wave optics, image signal processing, inverse reconstruction, computational sensing, and calibration. ImagingBench evaluates three complementary settings: Expert, fixed expert‑guided inverse reconstruction; Planner, planner‑guided inverse reconstruction; and Forward, forward‑system simulation for consistency checking. We benchmark leading proprietary and open‑source image‑centric multimodal systems, including Gemini, GPT, and Qwen, and compare them with representative task‑specific non‑agentic baselines. Across tasks, agentic models remain consistently weaker than specialized methods, especially on computational sensing problems such as lensless imaging, event‑based reconstruction, time‑of‑flight imaging, and holography. Planner guidance provides only modest and inconsistent gains over the fixed‑prompt Expert baseline. Although the models often generate visually plausible outputs, their reference‑based fidelity remains poor, revealing a substantial gap between semantic visual competence and physically grounded imaging performance. ImagingBench provides a unified testbed for measuring this gap and tracking progress in agentic AI for computational imaging.
Authors:Yi Yang, Myrna Castillo, Bodo Rosenhahn, Michael Ying Yang
Abstract:
Online 3D scene graph generation builds a persistent, structured representation of a scene by incrementally fusing 2D observations into a global 3D graph. Existing online methods treat this fusion as a fully deterministic pipeline, where we identify three sources of uncertainty that are overlooked: observation, 2D model, and 3D representation. We propose PUF: a Plug‑and‑play, Uncertainty‑aware, and training‑free Fusion framework. Scene graph node association is reformulated as a probabilistic likelihood over semantic and spatial factors, replacing binary accept/reject gates. Dirichlet evidence accumulation distributes class and relationship evidence across plausible candidates proportional to association likelihood. An optional class‑conditional prior completes edges for sparsely or never co‑observed object pairs. We instantiate PUF with both a 3D Gaussian and a 3D voxel backend and observe consistent improvements, demonstrating its ability to generalize across different representations. Experiments on the 3DSSG and ReplicaSSG benchmarks show that our method substantially outperforms existing approaches while maintaining real‑time latency. These results establish uncertainty‑aware fusion as a principled and effective paradigm for online 3D scene understanding. The source code is publicly available at https://github.com/yyyyangyi/PUF.
Authors:Xin-Jie Wu, Zhi-Hui You, Si-Bao Chen, Qing-Ling Shu, Xiao Wang, Jin Tang, Bin Luo
Abstract:
The core challenge of heterogeneous change detection in remote sensing imagery lies in effectively decoupling genuine land‑cover changes from significant modal disparities caused by distinct imaging mechanisms. These intrinsic inconsistencies are prone to introducing pseudo‑changes, thereby constraining detection accuracy. To address this, we propose a novel, end‑to‑end adversarial spatio‑frequency refinement network (ASFR‑Net). Initially, a modality‑invariant representation learner (MIR‑Learner) guides the backbone to extract modality‑invariant features, effectively bridging the primary domain gap. Subsequently, to address persistent residual modal differences, we design an innovative spatio‑frequency synergistic enhancement module (SFEM), which identifies and suppresses sensor‑specific noise and artifacts that are difficult to discern in the spatial domain by leveraging frequency‑domain processing. Multi‑level difference features are then computed from these refined representations and fed into a decoder equipped with cascaded hierarchical guided fusion module (HGFM) blocks to generate precise change maps. To alleviate the data scarcity in heterogeneous tasks, we construct and release a new high‑resolution benchmark specifically focused on building changes: the visible‑near‑infrared heterogeneous change detection (VisNIR‑HCD) dataset. It presents unique scientific challenges arising from deceptive visual similarity and non‑linear spectral inversions, providing a robust platform for evaluating model generalization. Extensive experiments on VisNIR‑HCD and public datasets demonstrate that ASFR‑Net achieves state‑of‑the‑art (SOTA) performance, significantly outperforming existing methods. The source code and the VisNIR‑HCD dataset are publicly available at https://github.com/LuoYang2024/ASFR‑Net.
Authors:Vladimir Gusev
Abstract:
The key‑value (KV) cache dominates the memory cost of long‑context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading. We study a complementary question: once a position's KV state has been quantized to codebook indices, how should the resulting symbol stream be stored, and can the storage layer do more than store? A family of contractive iterated‑map codes that serialize a symbol sequence into a sequence of low‑dimensional real vectors is revisited, and it is shown that they form a natural archive format for a quantized KV cache with the following features. The method provides exactly the access pattern a growing cache requires. It is lossless, it runs in linear time, and supports O(1) random access and O(1) amortized append. A controlled study of the quantizer feeding this archive is conducted on GPT‑2 with 1024‑token contexts. Keeping a small exact window (4 attention sinks + 32 recent tokens) and archiving the rest, per‑head residual vector quantization reduces the archived cache by 36‑54x relative to an fp16 cache at a perplexity cost of 11‑15%, and we quantify a sharp key/value asymmetry ‑‑ quantizing keys is roughly 4x more damaging than quantizing values, consistent with prior low‑bit KV work ‑‑ and use it to allocate bits in a hybrid scheme. Finally, we show the archive is simultaneously a search index: approximate substring queries execute directly on the stored vectors, and matched context is decoded from the matched vector without ever materializing the surrounding text. We release all code; every number reproduces from a single command on a laptop CPU.
Authors:Stepanida Alekseeva, Jenifer Kalafatovich, Seong-Whan Lee
Abstract:
In text‑to‑image in‑context learning (T2I‑ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query image. Recent studies show that state‑of‑the‑art multimodal large language models struggle with this setting, particularly due to limited compositional reasoning and sensitivity to prompt construction. In this work, we propose a Tree‑of‑Thoughts (ToT) reasoning framework for T2I‑ICL that introduces a multi‑stage reasoning and selection layer that generates, evaluates, and selects among multiple candidate hypotheses before constructing the final prompt for image synthesis. By exploring alternative reasoning branches and selecting a coherent interpretation, the proposed approach mitigates prompt ambiguity and compositional errors. We implement the proposed approach in a complete ToT‑T2IICL inference pipeline and evaluate it on the CoBSAT benchmark. Both qualitative and quantitative results show that structured multi‑branch reasoning leads to more consistent and semantically aligned image generation compared to baseline and Chain‑of‑Thought prompting strategies, without any additional training or fine‑tuning.
Authors:Guoyang Zhao, Quanhao Qian, Gongjie Zhang, Wenhao Li, Jiuniu Wang, Xiaowei Lu, Deli Zhao, Ran Xu
Abstract:
Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot's state within the scene, frequently underperforming even vision‑only baselines. To address this, we introduce GeoProp, a lightweight, plug‑and‑play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token. It then injects state‑derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short‑horizon predicted coordinate derived from recent kinematics, providing look‑ahead visual context. Across 67 tasks, GeoProp improves Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, and yields a 10.6% average gain across both policy families in the real world, while adding only 2‑3% to the parameter count. These results demonstrate that GeoProp is a simple yet high‑impact inductive bias for generalist embodied policies. Project page: https://alibaba‑damo‑academy.github.io/GeoProp/.
Authors:Zitong Andrew Chen, Junaid Hasan, Akhil Srinivasan, Hemkesh Bandi, Jarod Alper
Abstract:
Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition. In this work, we investigate how small transformers learn modular integer multiplication over composite moduli, a fundamentally non‑invertible operation due to the presence of zero‑divisors. We propose the monoid extension: a localized generalization of Group Composition via Representation (GCR) that suggests the learned computation does not rely on a single global representation space. Instead, the model partitions the input space into local hierarchical algebraic regions, where group‑like structure survives and Fourier mechanisms can be applied. In transformers trained on square‑free modular multiplication, we find that embeddings organize around these regions, attention exhibits class‑sensitive routing and low‑rank write directions, and local character features explain a large fraction of the model's output logits. Our results suggest that representation‑theoretic mechanisms previously identified for group operations can extend beyond groups to more general structures.
Authors:Nguyen Linh Dan Le, Nguyen Pham Hoang Le, Tran Dang Khoi
Abstract:
Medical image segmentation models can achieve strong benchmark performance while remaining sensitive to scanner, protocol, and institutional variation. These context shifts alter image appearance without changing the underlying lesion, allowing models to exploit nuisance cues that Dice and HD95 fail to expose. We present TRACE‑Seg3D, a counterfactual context auditing framework for robust 3D medical image segmentation. TRACE‑Seg3D preserves lesion‑relevant evidence and systematically varies imaging context to quantify prediction stability under controlled context shifts. The framework pairs each segmentation with audit evidence for context sensitivity and anatomical plausibility, enabling case‑level reliability assessment beyond overlap‑based evaluation. Experiments on BraTS and UTSW glioma segmentation benchmarks demonstrate competitive in‑distribution and cross‑domain performance. TRACE‑Seg3D also exposes context‑sensitive failure modes missed by conventional metrics. These results establish counterfactual context auditing as a practical route toward transparent and reliable 3D medical image segmentation under distribution shift. Our code is available at https://github.com/danleneurocom/Counterfactual‑Representation‑Network.
Authors:Kyuan Oh, Bumsoo Kim
Abstract:
Large vision‑language models incur substantial inference costs because high‑resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query. Existing pruning methods often combine query relevance and token diversity, yet these objectives can conflict under aggressive compression: relevance‑driven selection may overconcentrate the budget on correlated local evidence, while diversity‑driven selection may suppress indispensable tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training‑free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance‑ranked tokens, preserving a compact set of query‑critical evidence, and allocates the remaining budget through importance‑weighted novelty to recover informative, non‑redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture‑aware, and requires neither retraining nor model modification. Across image and video vision‑language models and benchmarks, it consistently improves the accuracy‑efficiency trade‑off over training‑free baselines, particularly under severe compression. On LLaVA‑NeXT‑7B, AnchorPrune preserves 97.6% of full‑token performance using only 160 of 2,880 visual tokens. These results establish relevance‑anchored contextual expansion as an effective principle for efficient multimodal inference. Code is available at https://github.com/MULTI‑cau/AnchorPrune.
Authors:Yujin Bae, Jaewoo Jeong, Hyeonseong Kim, Kuk-Jin Yoon
Abstract:
Anticipating human motion from an egocentric perspective is fundamental for proactive assistance in AR/VR, human‑robot collaboration, and embodied AI. While recent works incorporate language as a semantic prior to reduce the ill‑posed nature of egocentric forecasting, they largely neglect the 3D spatial and semantic context that governs how motion unfolds, and treat pose and language prediction as separate inference streams. We introduce Ego3DLM, built on two core principles: accurate motion forecasting requires explicit spatial and semantic understanding of the 3D environment, and pose and language must be predicted holistically in a single pass, since motion is inherently tied to the semantic interpretation of actions being performed. Given three‑point tracking, 3D scene features, and egocentric video, Ego3DLM simultaneously decodes past pose, future pose, past narration, and future narration in a single autoregressive pass, grounding predicted poses and descriptions in one another to enforce cross‑modal and temporal consistency. We adopt a three‑stage training scheme: (1) spatial‑semantic scene awareness pretraining; (2) holistic instruction tuning over all four outputs in a single pass; and (3) GRPO‑based reinforcement finetuning with intra‑ and inter‑modal rewards that directly optimize pose‑language fidelity. Experiments on the Nymeria benchmark demonstrate that Ego3DLM achieves state‑of‑the‑art performance across future motion prediction, past motion tracking, and motion description, showing that 3D scene grounding and holistic cross‑modal prediction yield physically plausible and semantically coherent motion forecasts. The project page is available at https://jaewoo97.github.io/Ego3DLM/.
Authors:Wenhao Feng, Yuxun Tang, Jiatong Shi, Qin Jin
Abstract:
Singing voice synthesis (SVS) has progressed rapidly, yet its ability to generalize across diverse musical genres remains underexplored. Existing benchmarks are heavily biased toward pop music, limiting systematic analysis of genre‑dependent behavior. We introduce MMGenre, a benchmark for multi‑genre SVS diagnosis, supported by an automatic pipeline for constructing genre‑aligned music scores. MMGenre spans 10 major genres and 26 subgenres, enabling comprehensive analysis of genre‑aware synthesis. Extensive evaluation of representative SVS models reveals limited genre discrimination: synthesized vocals across genres exhibit highly similar acoustic characteristics and weak separability. While zero‑shot genre adaptation yields only marginal improvements, lightweight genre‑specific continued training leads to substantial gains. MMGenre provides a standardized framework for multi‑genre SVS evaluation and exposes critical challenges in achieving genre‑aware singing voice synthesis.
Authors:Hao Kong, Di Liu, Shuo Huai, Xiangzhong Luo, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu
Abstract:
Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks. However, the prohibitive computational cost of CNNs hinders the deployment of CNNs onto resource‑constrained embedded devices. To address this issue, we propose EdgeCompress, a comprehensive compression framework to reduce the computational overhead of CNNs. In EdgeCompress, we first introduce dynamic image cropping (DIC), where we design a lightweight foreground predictor to accurately crop the most informative foreground object of input images for inference, which avoids redundant computation on background regions. Subsequently, we present compound shrinking (CS) to collaboratively compress the three dimensions (depth, width, and resolution) of CNNs according to their contribution to accuracy and model computation. DIC and CS together constitute a multidimensional CNN compression framework, which is able to comprehensively reduce the computational redundancy in both input images and neural network architectures, thereby improving the inference efficiency of CNNs. Further, we present a dynamic inference framework to efficiently process input images with different recognition difficulties, where we cascade multiple models with different complexities from our compression framework and dynamically adopt different models for different input images, which further compresses the computational redundancy and improves the inference efficiency of CNNs, facilitating the deployment of advanced CNNs onto embedded hardware. Experiments on ImageNet‑1K demonstrate that EdgeCompress reduces the computation of ResNet‑50 by 48.8% while improving the top‑1 accuracy by 0.8%. Meanwhile, we improve the accuracy by 4.1% with similar computation compared to HRank, the state‑of‑the‑art compression framework. The source code and models are available at https://github.com/ntuliuteam/edge‑compress
Authors:Seulbin Hwang, Kiyoung Om, Daejung Kim, Jinhan Lee
Abstract:
Realistic and diverse traffic simulation is essential to autonomous driving development. Yet prevailing benchmarks predominantly reward realism, and recent methods have optimized accordingly, leaving diversity underexplored. We introduce Flow‑ERD, a multi‑agent simulator that pursues realism and diversity jointly. Its backbone, Agent‑Type Aware Flow Matching (AFM), couples flow matching's multi‑modal expressiveness with type‑specific kinematic execution. It preserves fine‑grained diversity while keeping motions consistent with each agent type. A second stage, Entropy‑Regularized Distillation (ERD), fine‑tunes the closed‑loop rollout distribution with an entropy‑regularized reverse‑KL objective. This mitigates covariate shift while explicitly preventing collapse onto high‑density modes. We evaluate Flow‑ERD with a log‑free diversity metric alongside standard realism scores. Flow‑ERD ranks first on the WOSAC test benchmark and dominates the realism‑‑diversity Pareto front among reproducible baselines. Our project page is available \hrefhttps://seulbinhwang.github.io/flow‑erd‑project‑page/here.
Authors:Xiangyu Meng, Shicai Wei
Abstract:
Multimodal learning robust to missing modalities is essential for real‑world applications. Existing methods mainly focus on inter‑modality missing, where entire modalities are absent, while overlooking intra‑modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter‑modality missing and intra‑modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation‑aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise‑aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise‑Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu‑Five/GIML.
Authors:Chuyao Zhang, E Li, Taochen Chen, Yiqun Zhang, Yuzhu Ji, Shuping Zhao, Peng Liu, Yiu-ming Cheung
Abstract:
Missing data is prevalent in practical applications, making effective imputation an essential preprocessing step for downstream analysis. Real‑world datasets often exhibit complex latent structures composed of multiple subgroups with distinct distributions. However, existing methods often overlook such population heterogeneity. Without explicit structural guidance, these methods tend to produce generic estimates that blur subgroup boundaries and lack instance‑level fidelity. While incorporating subgroup information offers a remedy, it faces a circular dependency: reliable subgroup identification requires complete data, while data completion is the imputation objective itself. To resolve this, we propose CAGI (Cluster‑Aware Generative Imputation), a framework that reformulates clustering and imputation as a mutually reinforcing co‑optimization process. CAGI employs a ``Partition‑Guide‑Restore'' strategy where dynamic cluster assignments act as local priors to condition a Generative Adversarial Network. An iterative feedback loop is established to progressively refine both cluster structures and imputed values toward faithful subgroup distributions. To ensure distributional stability, CAGI further employs a multi‑level optimization objective combining instance‑level reconstruction with distribution‑level regularization. Extensive experiments on 14 benchmark datasets with 15 representative baselines demonstrate the superiority of CAGI. The source code is available at: https://github.com/supercocachii/CAGI
Authors:Sirui Zhang, Tianle Wang, Xinyi Tong, Peiyang Yu, Jishang Chen, Liangke Zhao, Haoxin Zhang, Duo Xu, Xin Jin, Feng Yu, Songchun Zhu
Abstract:
Music aesthetic assessment is a challenging yet underexplored problem, requiring models to capture fine‑grained, multi‑dimensional human perceptual judgments. Progress in this area has been limited by the lack of large‑scale datasets with structured aesthetic annotations. We introduce MADB, a large‑scale dataset and benchmark comprising 9,999 tracks annotated by 30 trained annotators. Each track is rated by around 10 annotators across 10 perceptual dimensions and one overall score, with additional textual comments for multimodal analysis. We establish a unified evaluation framework over multiple pretrained models. Results reveal substantial gaps between model predictions and human judgments, exposing key limitations of current approaches. MADB provides a new benchmark for human‑aligned music understanding. Project page: https://github.com/knownree/madb
Authors:Shuo Huai, Di Liu, Hao Kong, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin
Abstract:
Deep learning applications have been widely adopted on edge devices, to mitigate the privacy and latency issues of accessing cloud servers. Deciding the number of neurons during the design of a deep neural network to maximize performance is not intuitive. Particularly, many application scenarios are real‑time and have a strict latency constraint, while conventional neural network optimization methods do not directly change the temporal cost of model inference for latency‑critical edge systems. In this work, we propose a latency‑oriented neural network learning method to optimize models for high accuracy while fulfilling the latency constraint. For efficiency, we also introduce a universal hardware‑customized latency predictor to optimize this procedure to learn a model that satisfies the latency constraint by only a one‑shot training process. The experiment results reveal that, compared to state‑of‑the‑art methods, our approach can well‑fit the 'hard' latency constraint and achieve high accuracy. Under the same training settings as the original model and satisfying a 34 ms latency constraint on the ImageNet‑100 dataset, we reduce GoogLeNet's latency from 40.32 ms to 34 ms with a 0.14% accuracy reduction on the NVIDIA Jetson Nano. When coupled with quantization, our method can be further improved to only 0.04% drop for GoogLeNet. On the NVIDIA Jetson TX2, we compress VGG‑19 from 119.98 ms to 34 ms and even improve its accuracy by 0.5%, and we scale GoogLeNet up from 20.27 ms to 34 ms and achieve higher accuracy by 0.78%. We also open source this framework at https://github.com/ntuliuteam/ZeroBN
Authors:Paul F. R. Wilson, Mohamed Harmanani, Zhuoxin Guo, Obed K. Dzikunu, Hannes Cash, Adam Kinnaird, Brian Wodlinger, Purang Abolmaesumi, Parvin Mousavi
Abstract:
Artificial intelligence (AI) analysis of micro‑ultrasound (μUS) has shown promise for prostate cancer (PCa) detection. However, most existing AI methods focus on the analysis of single μUS images in isolation. By contrast, expert μUS readers typically assess a full recorded video study, which provides three‑dimensional context, to improve PCa detection compared to single‑frame analysis. Inspired by this clinical workflow, we propose Compass, a novel AI methodology which models a μUS study as a stream of 2D images. Compass jointly integrates rotational sweep videos of the prostate with μUS frames acquired at the moment of biopsy, and performs evidence aggregation across the study using a transformer conditioned on the probe's rotational angle. Finally, a decoder head predicts frame‑level and study‑level risk scores for the patient. The model is trained and evaluated using a multi‑center clinical trial dataset of μUS studies, including continuous rotational scans of the prostate and videos captured during biopsy acquisition. We compare the proposed method to baseline AI methods from the literature and to risk scores provided by clinical experts. Our framework shows strong performance, highlighting the value of multi‑view context for μUS PCa detection, and providing a potentially powerful tool to complement human expertise in μUS‑based PCa diagnosis. Our code is available at: https://github.com/mharmanani/Compass.
Authors:Hao Kong, Di Liu, Shuo Huai, Xiangzhong Luo, Weichen Liu, Ravi Subramaniam, Christian Makaya, Qian Lin
Abstract:
Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI. However, as an image usually contains much spatial redundancy, e.g., background pixels, directly shrinking the whole image will lose important features of the foreground object and lead to severe accuracy degradation. In this paper, we propose a dynamic image cropping framework to reduce the spatial redundancy by accurately cropping the foreground object from images. To achieve the instance‑aware fine cropping, we introduce a lightweight foreground predictor to efficiently localize and crop the foreground of an image. The finely cropped images can be correctly recognized even at a small resolution. Meanwhile, computational redundancy also exists in CNN architectures. To pursue higher execution efficiency on resource‑constrained embedded devices, we also propose a compound shrinking strategy to coordinately compress the three dimensions (depth, width, resolution) of CNNs. Eventually, we seamlessly combine the proposed dynamic image cropping and compound shrinking into a unified compression framework, Smart Scissor, which is expected to significantly reduce the computational overhead of CNNs while still maintaining high accuracy. Experiments on ImageNet‑1K demonstrate that our method reduces the computational cost of ResNet50 by 41.5% while improving the top‑1 accuracy by 0.3%. Moreover, compared to HRank, the state‑of‑the‑art CNN compression framework, our method achieves 4.1% higher top‑1 accuracy at the same computational cost. The codes and data are available at https://github.com/ntuliuteam/smart‑scissor
Authors:Trang Nguyen, Sidong Zhang, Shiv Shankar, Gauri Jagatap, Deepak Chandran, Andrea Fanelli, Madalina Fiterau
Abstract:
Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reaction prediction and other content engagement applications, we introduce Video2Reaction, a multimodal dataset that maps short movie segments to a distribution of induced emotions of viewers in the wild, as expressed through social media. Video2Reaction spans more than 10,000 videos and serves as a reliable benchmark as well as a training resource for audience reaction prediction. To enable cost‑effective continuous annotations as reactions may change over time, we develop a two‑stage multi‑agent pipeline using only open‑source LLMs, achieving 86% correctness under blind human verification despite the inherently noisy and subjective nature of the task. We establish the first benchmark for video‑to‑reaction‑distribution prediction in the wild and show that pretrained foundation video models fail in zero‑shot settings, while finetuning transforms them into state‑of‑the‑art predictors capable of modeling both full reaction distributions and dominant responses from video alone. However, the task remains challenging: even the strongest methods achieve only 77% Top‑3 F1 in dominant reaction prediction (LLaVA‑Next), highlighting a substantial gap in modeling collective audience reaction. \modificationDataset and code are available at our project page: https://information‑fusion‑lab‑umass.github.io/video2reaction‑bench.github.io
Authors:Nima Kelidari, Mohammadsaeed Haghi, Mahdi Salmani
Abstract:
Reinforcement learning agents for imperfect‑information card games are only as strong as the opponents they train against, and they are hard to grade, since they beat a random opponent over 99 percent of the time and only tie copies of themselves. So we build a strong, fixed, rule‑based expert for Gin Rummy and use it only as a yardstick, never for training. It beats every agent we trained 70 to 99 percent of the time. Across more than a hundred runs, we isolate what makes a lightweight agent stronger. Trust region updates, a well‑aimed reward, a curriculum of tougher opponents, warm starting, and keeping the best checkpoint all help, and stacking them lifts a self‑play champion from about 30 to 36 percent against the expert. Several ideas did not pay off. Short‑term and longer‑term reward shaping, learned state embeddings, imitation and DAgger, and a live large language model opponent were each unhelpful, too slow, or too heavy to train at scale. Comparing MLP, convolutional, set‑based, attention, and recurrent encoders shows that extra capacity does little to break the ceiling, suggesting the limit is information rather than network size. We add standard baselines (neural fictitious self‑play and information set Monte Carlo search) and confirm the approach carries over to Leduc Hold'em, where the optimum is computable. The result is a lightweight, game‑agnostic recipe that trains competitive agents without training on the expert, for any game a small model can handle, reported with robust statistics and released as a reusable package.
Authors:Sakuya Ota, Qing Yu, Kent Fujiwara, Satoshi Ikehata, Ikuro Sato
Abstract:
Diffusion‑based text‑to‑motion models synthesize realistic human motions but often exhibit semantic drift from the input text. Motion is inherently temporal, especially in compositional and long‑duration sequences that require semantic consistency across multiple action segments and smooth kinematic transitions throughout the trajectory. We posit that the initial noise is central to this consistency: within the Gaussian noise space, certain instances, i.e. winning noise tickets, carry latent structure that biases denoising toward particular motion semantics, even under null prompts. We propose WInning Noise Retrieval and Optimization (WINRO), a training‑free, model‑agnostic framework that improves text‑motion alignment by selecting and refining such tickets before diffusion sampling. WINRO maps random noises to motion features generated under null prompts, retrieves the best‑aligned noise for a given text, and refines it via a KL‑regularized objective that reduces the residual semantic gap while preserving the Gaussian prior. An optional LoRA‑based adapter amortizes this refinement into a single forward pass. WINRO consistently improves text‑motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining, improves temporal robustness on the MTT benchmark, and generalizes to applications such as motion stylization and spatial constraint satisfaction.
Authors:Xiangyu Han, Mengyu Yang, Jiaqi Li, Bowen Chang, Ziyu Chen, Hexu Zhao, Rahul Kumar Agrawal, Anthony Rodriguez, Fiona Hua, Marco Pavone, Chen Feng, Yiming Li
Abstract:
Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable scale? Although recent foundation models have advanced scene reconstruction and embodied intelligence, scaling to entire cities remains an open challenge, primarily due to the lack of city‑scale data. To bridge the gap, we introduce WildCity, a real‑world multimodal dataset collected by autonomous fleets traversing complex urban environments. Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in‑the‑wild perception, e.g., dynamic objects, lighting variations, and imperfect camera poses. We further establish an urban‑tailored reconstruction baseline and convert the reconstructed environments into a closed‑loop simulator. Beyond the dataset and baseline, we systematically analyze the key challenges on the path to simulation‑ready urban digital twins: scalability, extrapolation, and uncertainty. Ultimately, WildCity aims to catalyze progress not only in city‑scale rendering, but more broadly in the pursuit of AI that can perceive, remember, and reason across space at a scale comparable to human cognition. Project page: https://han‑xiangyu.github.io/Wild‑City/
Authors:Javidan Abdullayev, Maxime Devanne, Jonathan Weber, Germain Forestier
Abstract:
Deep learning has achieved remarkable success in various domains including time series analysis, computer vision and natural language processing. However, high computational and memory demands of state‑of‑the‑art architectures pose challenges for deployment in resource‑limited environments. Knowledge Distillation (KD) addresses this by transferring knowledge from a large teacher model to a smaller, more efficient student model while maintaining competitive performance. In this work, we investigate the effectiveness of KD for Time Series Classification (TSC) across three architectures: the classical Fully Convolutional Network (FCN), the convolutional Inception model and the transformer‑based ConvTran model. We evaluate our approach on UCR Archive, the largest benchmark repository of time series datasets, by modifying architectural components such as convolutional filters, Inception modules and attention heads across the three architectures. Our results consistently show that KD most effectively benefits student models of intermediate complexity across all three architectures, with the distilled FCN student reducing parameters by a factor of 38, the distilled Inception student achieving nearly the same performance as the teacher with 42% fewer parameters and the distilled ConvTran student with 2 attention heads showing the most significant improvement through distillation. To encourage further research and reproducibility, we provide our implementation at https://github.com/MSD‑IRIMAS/KD‑4‑TSC.
Authors:Feng Xia, Shuo Zhang, Xi Wang
Abstract:
Conversational Recommender Systems (CRSs) are interactive systems that use multi‑turn natural language dialogue to understand evolving user preferences and provide personalized recommendations. To achieve this goal, CRSs rely on preference elicitation strategies to actively gather informative preference cues from users; however, the timing and selection of these strategies during a conversation remain largely unexplored. While many existing studies emphasize eliciting explicit item attributes and tend to adopt relatively static elicitation strategies, the use of item‑based preference elicitation and how it varies across different dialogue stages remains less explored. In this work, we conduct a systematic investigation of preference elicitation strategies from a stage‑aware perspective. We provide empirical evidence that optimal preference elicitation strategies are stage‑dependent and context‑sensitive: attribute‑based inquiries are effective in early stages, while item‑based strategies become superior as preferences refine. To support this paradigm, we introduce InPE, a dataset enriched with fine‑grained annotations for elicitation necessity and strategy selection. With this dataset, we propose COPE (COnversational Preference Elicitation via Mixture of Experts), a novel architecture for strategy modeling. Extensive offline evaluation on our dataset indicates that context‑aware preference elicitation strategies are beneficial for conversational recommendation. In addition, the analysis of the predicted strategies uncovers consistent stage‑wise tendencies in dialogue progression, providing empirical evidence of common interaction patterns in conversational recommendation systems. Our dataset is available at https://github.com/juanfacabian/InPE.
Authors:Sankalp Gilda
Abstract:
Finance, sensing, and demand streams violate the exchangeability that IID conformal prediction and the IID bootstrap assume, and existing libraries implement either a general resampling engine or conformal calibration without the other. tsbootstrap provides block, residual, sieve, and wild resampling, classical bootstrap confidence intervals, and adaptive conformal calibrators (EnbPI, ACI, NexCP, AgACI) through a single typed API in which a specification object selects each method. In a controlled coverage study the IID bootstrap undercovers sharply under dependence; dependence‑aware methods reduce the coverage deficit, the sieve nearest to nominal under short‑memory linear dependence. On the shared fixed‑statistic path a compiled backend runs several times faster than arch, and a streaming reduce avoids materializing the O(Bn) replicate tensor, limiting peak extra memory to O(B) for the statistic array. The software is MIT licensed (v0.6.1).
Authors:Niels Cariou-Kotlarek, Vasileios Lampos
Abstract:
Rough path signatures are a universal feature map for continuous paths and, via the expected signature, characterise path distributions. These guarantees do not directly extend to cadlag paths of Temporal Point Processes (TPPs), limiting the use of signature methods for event sequences. Furthermore, neural TPP models, including recent generative approaches, optimise per‑event objectives with no global sequence‑level loss, while evaluation of variable‑length event sequences lacks distributional discrepancy measures. This paper proposes a common pathwise framework for addressing these limitations. We introduce the interarrival embedding, a stable, injective lift from jump paths to continuous paths of bounded variation, extending signature methods to discrete event sequences. Our theoretical contributions give rise to sigTPP, the first signature‑based generative model for TPPs, trained using a path‑level loss on complete trajectories. We further analyse the space of counting paths and derive three distributional discrepancies, providing mathematically justified tools for evaluating generative TPP models. Across synthetic and real‑world datasets, sigTPP achieves the best average rank based on eight complementary metrics, outperforms or is within a standard error of the strongest baseline in 64% of the dataset‑metric pairs, and according to a relative score, improves against every baseline by at least 19% on average.
Authors:Yifan Zhang, Yuxin Hu, Zhuobin Hao, Xiaozhuan Gao, Lipeng Pan
Abstract:
Self‑paced learning (SPL) is an effective learning paradigm that simulates the human learning process by progressing from easy to difficult samples based on the value of the loss function during the learning process. It has shown great potential in improving model performance and training efficiency. However, the prediction results of samples with smaller loss values are not necessarily reliable, indicating that such samples are not always simple samples for the model. Hence, this article proposes an uncertainty‑aware self‑paced learning based on evidential neural networks, termed UASPL, which integrates predictive reliability into sample selection through a general loss function within the Subjective Logic framework. This loss function incorporates uncertainty estimation and can be extended to different variants of SPL. Moreover, this loss function couples a sample selection preference, thereby ensuring the interpretability of the sample selection process. Finally, the experimental results on multiple datasets show that UASPL outperforms other SPL methods in terms of classification performance, interpretability, and generality. The source code is available at: https://github.com/treelife979/UASPL.
Authors:Fabien Polly
Abstract:
Compact networks built from Clifford algebra Cl(3,0) primitives are exactly SO(3)‑equivariant and learn synthetic 3D vector laws from few samples. We ask whether the geometric algebra structure itself contributes anything beyond exact equivariance. We compare against a minimal scalarization baseline: invariant dot products fed to a small MLP that outputs coefficients on the equivariant basis v_i, v_i x v_j, which is also exactly equivariant. On single‑stage laws (rotation by axis‑angle, cross product, central force), scalarization matches or beats the Cl(3,0) network at a fraction of the training cost, so the geometric algebra adds nothing there. On compositional targets whose computation graph nests group operations (apply R2 R1 to a point; map a local force through an orientation, then take a torque), the Cl(3,0) network beats scalarization by an order of magnitude in the low‑data regime, reaching with 100 samples what the baseline needs 3000 for, and the gap survives strengthening the baseline with the triple‑product invariant and 17x more parameters, external Vector Neurons and e3nn baselines, and a multiplicative coefficient network. Ablations show the required network depth tracks the rotation chain length, and scalarization falls below the constant predictor on chains of four rotations. The advantage is not composition per se: on a rotation‑free nested cross product, which flattens into polynomial invariant coefficients, scalarization wins by 24x. No tested model, equivariant or not, extrapolates invariant magnitudes: on radius and separation shifts every model is worse than a constant predictor once errors are normalized. We conclude that geometric algebra layers are not a general shortcut for low‑data 3D learning, but become useful precisely when the target composes group elements in depth.
Authors:Andrey Podivilov, Vadim Lomshakov, Sergey Savin, Matvei Startsev, Roman Pozharskiy, Maksim Parshin, Sergey Nikolenko
Abstract:
We present AgentLens, a production‑assessed benchmark for interactive code agents. Most code‑agent benchmarks reduce a run to a single bit ‑‑ did the task pass? ‑‑ but the people who actually use these agents experience the entire trajectory: how the agent follows instructions, uses its tools, verifies its own work, recovers from mistakes, and talks to them along the way. AgentLens evaluates that whole trajectory. It pairs formal verification, where an objective check exists, with LLM‑written trajectory reviews and side‑by‑side comparisons, so that each run yields a readable explanation of why the score is what it is. This makes AgentLens useful for more than ranking models: we use it to diagnose model behavior, compare successive versions of our own agent, and catch product regressions in a nightly evaluation pipeline. We release the benchmark as open source at https://github.com/agent‑lens/agent‑lens‑bench.
Authors:Li Hengyu
Abstract:
The pre‑softmax score of an attention head is a bilinear form score(i,j) = x_i^T M x_j in a learned operator M = W_q^T W_k. Because M is generally non‑symmetric, hence non‑normal, it has a complex eigenspectrum and non‑orthogonal eigenvectors, the regime where non‑Hermitian and random‑matrix tools apply. We ask what this spectrum encodes, at three levels for previous‑token and induction circuits. Statically, across seven pretrained models spanning three positional schemes, the strongest previous‑token heads are spectrally rotational under RoPE and non‑rotational, or content‑like, where position enters outside QK (learned‑absolute and ALiBi); the model‑level separation is perfect at every top‑k examined (exact permutation p=0.029), and zeroing the per‑frequency RoPE phase Im(M_t) eliminates induction on a pre‑identified previous‑token head in all three RoPE models. Dynamically, over public Pythia checkpoints every head originates at the random‑matrix (Ginibre) null; the rotational signature emerges with the behavior, not before it, and the population‑median suppression that yields the final profile follows circuit formation, so the profile is a consolidated fingerprint, not a precursor. Causally, and at toy scale, no spectral channel is necessary: constrained two‑layer training reroutes around every ban with capability intact, albeit at a significant formation delay (four pre‑registered contrasts, q_BH <= 0.016). The cost structure exposes each scheme's default: imposing symmetry slows learned‑absolute models by a factor of 2.9, whereas a RoPE head with a fully symmetric static M still routes directionally via the phase channel, impossible under absolute positions. Within the settings examined, the positional scheme sets the default spectral algebra of an attention head's solution: a fingerprint sculpted after function, not a hard constraint upon it.
Authors:Andrei-George Durdun, Victor Constantinescu, Radu Tudor Ionescu
Abstract:
Automatically recognizing the sentiment, positive or negative, from speech is a challenging task, requiring both the analysis of vocal inflections and the interpretation of uttered words. Recent solutions rely on audio foundation models to solve the task, but it remains unclear if such models can take all aspects into account. To this end, we propose a multimodal solution that integrates audio and text information via cross‑modal transformers, where text transcripts are automatically generated via an automatic speech recognition (ASR) tool. Moreover, we create multiple text modalities by automatically translating the transcripts into multiple languages via machine translation tools. Audio and multilingual text features are combined via a cascaded architecture comprising cross‑modal transformer blocks that integrate modalities one by one. We further distill knowledge from the multimodal model, called teacher, into a unimodal (audio only) model, called student. We conduct experiments on a large‑scale dataset, demonstrating that the automatically generated textual information can bring significant performance boosts in multimodal sentiment polarity classification. Our ablation study confirms that both automatic transcripts and automatic translations are helpful. Moreover, we show that the audio‑only model can be enhanced via distillation, boosting performance without any computational overhead during inference. To reproduce the reported results, we publicly release our code at https://github.com/andreidurdun/cross‑modal‑audio‑sentiment.
Authors:Lanhao Li, Bingshu Xie, Lijun Sun, Xin Xue, Haoyi Zhou, Jianxin Li
Abstract:
Accurate long‑term forecasting in complex systems is frequently compromised by dataset‑level distribution shifts, where diverse underlying behavioral modes and evolving system states drive the dynamic multivariate time‑series. While existing methods predominantly focus on local temporal shifts, they fail to explicitly model the global structural challenge where datasets are composites of distinct operational regimes. In this paper, we propose NEST, a specialized framework designed to model and recompose these evolving structures through a two‑phase dense MoE architecture. NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes through unsupervised clustering in a principled moment‑entropy space. We introduce a regime‑oriented router mechanism that generates initial expert weights based on temporal content, subsequently refined through geometric modulation to regime centroids. Crucially, rather than acting as monolithic predictors, individual experts function as specialized kernels that capture regime‑specific dynamics by evolving unique variate‑attention patterns. Extensive evaluations on diverse benchmarks, including heterogeneous network traffic and physical phenomena, demonstrate that NEST consistently achieves state‑of‑the‑art performance. Our code and datasets are available at https://github.com/Aaralshin/NEST
Authors:Haoyu Zhao, Xingyue Zhao, Siteng Huang, Xin Li, Deli Zhao, Zhongyu Li
Abstract:
Robotic manipulation in the open world requires not only recognizing what a scene looks like, but also anticipating how its 3D structure moves under interaction. We argue that synchronized RGB, depth, and optical flow, namely RGB‑DF, provide a physically grounded representation that captures the underlying 4D dynamics of a scene. Compared to 2D pixel videos, this multi‑modal synergy aligns visual appearance with geometric structure and temporal motion, creating a representation space significantly closer to the low‑level end‑effector actions demanded by robotic systems, thereby narrowing the gap between world prediction and policy learning. Building on this insight, we introduce RynnWorld‑4D, a generative model that co‑produces future RGB frames, depth maps, and optical flow from a single RGB‑D image and a language instruction within one unified diffusion process. This 4D world model features a tri‑branch architecture that integrates cross‑modal attention with frame‑wise 3D RoPE, ensuring that appearance, geometry, and motion evolve consistently. To supply training data at scale, we curate Rynn4DDataset 1.0, a massive dataset of over 254.4 million frames across egocentric human and robotic manipulation videos with high‑quality pseudo‑labels for depth and optical flow. We further propose RynnWorld‑4D‑Policy, an inverse dynamics head that consumes the internal 4D representations of RynnWorld‑4D in a single forward pass, bypassing expensive multi‑step denoising, to output robot actions in a closed‑loop manner. Experiments show that RynnWorld‑4D produces temporally and spatially coherent 4D predictions, and that RynnWorld‑4D‑Policy achieves state‑of‑the‑art performance on real‑world dexterous bimanual manipulation tasks, particularly excelling in tasks demanding spatial precision and temporal coordination.
Authors:Haoyu Zhao, Xingyue Zhao, Hangyu Li, Biao Gong, Kehan Li, Siteng Huang, Xin Li, Deli Zhao, Zhongyu Li
Abstract:
Scaling robot learning requires massive, diverse trajectory data, yet collection is currently bottlenecked by physical teleoperation, where every demonstration binds operator time to specific hardware and workspaces. We introduce digital teleoperation, a paradigm that decouples data collection from physical constraints by replacing the real robot with a generative world model. In this framework, an operator's hand‑pose stream drives a robot‑centric generative world model to synthesize high‑fidelity egocentric videos from a single reference image. The recorded pose stream serves as an embodiment‑agnostic action label transferable to any target robot via standard retargeting, yielding complete state‑action trajectories for imitation learning independent of physical hardware. We instantiate this paradigm in RynnWorld‑Teleop, a system that integrates depth‑aware skeletal conditioning, progressive human‑to‑robot training on a video Diffusion Transformer, and streaming autoregressive distillation. This pipeline compresses the generative process into a single‑pass inference, enabling 40+ FPS, real‑time interactive generation on a single H100 GPU. Policies trained exclusively on RynnWorld‑Teleop‑generated data achieve effective zero‑shot Sim2Real transfer across dexterous and diverse bimanual tasks. Moreover, augmenting real‑world datasets with our digitally teleoperated data consistently improves success rates, demonstrating that RynnWorld‑Teleop serves as a high‑fidelity, scalable data engine for the next generation of robotic agents.
Authors:Ruihang Zhang, Felix Taubner, Pooja Ravi, Kiriakos N. Kutulakos, David B. Lindell
Abstract:
Tracking the six‑degree‑of‑freedom (6‑DoF) pose of objects and surfaces from monocular video is a long‑standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself‑such as 3D models, depth maps, object masks, or task‑specific learned features‑and they struggle with textureless, transparent, reflective, or deformable surfaces. Here, we introduce ProxyPose, which recasts 6‑DoF pose tracking as video‑to‑video translation. Given only a video and a single marked pixel in the first frame, a fine‑tuned video diffusion model translates the input into a proxy video‑a synthetic video depicting a colored polyhedron undergoing the same local rigid‑body motion as the surface region at the marked pixel. Because the proxy's geometry and appearance are known by construction, recovering its full 6‑DoF trajectory reduces to classical pose estimation with off‑the‑shelf solvers. This formulation leverages large‑scale video pre‑training to absorb the hardest aspects of pose tracking‑handling challenging materials, occlusions, and deformations‑into the translation step, while operating at the pixel level with no assumptions about object identity, boundaries, or global rigidity. ProxyPose achieves state‑of‑the‑art 6‑DoF pose tracking accuracy without the additional inputs required by competing methods and after fine‑tuning the video model only on synthetic data. We further demonstrate that ProxyPose extends to face tracking, camera pose estimation, and challenging in‑the‑wild scenes that are beyond the reach of existing approaches. Project page: https://ruihangzhang97.github.io/proxypose/.
Authors:He Liang, Chenyang Ma, Yiming Zhang, Sangyun Shin, Andrew Markham, Niki Trigoni, Yuhang He
Abstract:
Existing 3D scene‑grounded Large Language Models (3D‑LLMs) focus on answering questions grounded in simplified single‑room 3D scenes, lacking the ability to reason over real‑world household environments containing multiple interconnected rooms and diverse object categories. We introduce CAIRN, a topology‑aware 3D‑LLM for multi‑room 3D scene understanding. CAIRN aligns transformer attention with scene hierarchy, giving the model explicit awareness of object‑level relations and room‑level connectivity. It enriches object tokens with room‑local relational context via a graph neural network, introduces learned room tokens for room‑level abstraction, and applies a hierarchical attention mask with geometric bias to route information according to scene topology. CAIRN is developed on CAIRN‑MR, a benchmark we introduce on HM3D for multi‑room 3D scene understanding, covering grounding, captioning, and four question‑answering tasks that progressively evaluate from intra‑room perception to cross‑room reasoning. Experiments show that CAIRN outperforms prior 3D‑LLMs by a large margin across all CAIRN‑MR tasks while remaining competitive on five single‑room benchmarks.
Authors:Songbur Wong, Xiaosong Jia, Junqi You, Bo Zhang, Pei Xu, Renqiu Xia, Yuping Qiu, Shaofeng Zhang, Zelin Zhao, Xuechao Yan, Yuchen Zhou, Yurui Chen, Wen Guo, Hang Xu, Junchi Yan
Abstract:
Evaluating end‑to‑end autonomous driving (E2E‑AD) remains challenging, as existing driving simulation methods often trade off closed‑loop interactivity (e.g., CARLA) and real‑world visual fidelity (e.g., nuScenes). We present \emphPoint as Skeleton, a generative sensor simulation framework for state‑updated autoregressive driving video generation, in which an autoregressive generator synthesizes visual observations from step‑wise updated ego states, actor states, scene maps, and point‑cloud skeleton conditions. To support closed‑loop rollout, we introduce Reset‑and‑Roll, which adapts rolling diffusion inference to simulation by preventing future‑conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step‑wise autoregressive rollout, we introduce point‑cloud skeletons that decouple foreground and background assets and project them into camera‑view painted‑point and template‑depth conditions, providing appearance and geometric cues. We further implement a nuPlan‑based renderer‑level closed‑loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that Point as Skeleton improves autoregressive generation quality during closed‑loop rollout, demonstrating its potential for visually faithful closed‑loop driving simulation. The code is available at https://github.com/krauwu/point‑as‑skeleton.
Authors:Chase McDonald, Nathan Tsang, Wesley N. Kerr
Abstract:
We present FootsiesGym, an open‑source environment for learning in a non‑trivial two‑player, zero‑sum, imperfect‑information game. Built on HiFight's minimalist 2D fighting game Footsies, it isolates the cyclic, non‑transitive strategic interactions of fighting game neutral play while remaining simple enough for efficient analysis. We provide a vectorized simulator that enables high‑throughput training on standard hardware, making the environment accessible and reproducible. We describe the design of the environment, benchmark several reinforcement learning algorithms, and discuss open research directions it enables. The code is available at https://github.com/como‑research/FootsiesGym.
Authors:Przemysław Rola
Abstract:
We introduce EntroPath, a manifold learning method that recovers geodesic geometry from data graphs through ensembles of diffusion paths. Many existing graph‑based embeddings rely either on locally normalised random walks or on shortest‑path distances. The former can concentrate diffusion in densely sampled regions, while the latter are sensitive to spurious shortcut edges in the graph. EntroPath instead builds its dissimilarities from the maximum entropy random walk (MERW), which aggregates the full ensemble of k‑step paths between points rather than relying on any single trajectory. We show that the resulting free‑energy dissimilarity converges to squared geodesic distance in the short‑time limit, via Varadhan's heat‑kernel formula. The diffusion depth k interpolates smoothly between local neighbourhood structure and global manifold geometry, and the symmetrised kernel admits an exact Gram factorisation connecting EntroPath to kernel methods. We further provide scalable extensions via landmark projection and diffusion‑potential pseudotime. Across synthetic manifolds and single‑cell benchmarks, EntroPath consistently matches or outperforms diffusion‑ and shortest‑path‑based methods, while remaining competitive with neighbourhood‑preserving embeddings (UMAP, t‑SNE) on local‑structure metrics. Its gains are most pronounced on manifolds with non‑uniform sampling density and well‑separated branching trajectories, where path‑ensemble diffusion more faithfully preserves the underlying geodesic geometry.
Authors:So Hasegawa, Shailaja Keyur Sampat, Lei Liu, Wei-Peng Chen
Abstract:
Current benchmarks for evaluating Large Language Models (LLMs) in data analysis often fail to reflect real‑world settings. They typically focus on fact retrieval from small tables and overlook the challenges of large multi‑tabular datasets, external knowledge integration, and exploratory insight discovery. We introduce DataGovBench, a benchmark derived from governmental open data designed to evaluate LLMs in practical scenarios. The benchmark includes two tasks: Table QA that requires solving complex decomposable questions and producing textual answers or visualizations, and Table Insight that evaluates the ability of models to generate expert‑level findings through exploratory data analysis. Comprehensive experiments with state‑of‑the‑art LLMs, both with and without agentic frameworks, reveal significant performance gaps across both tasks. These results suggest that current LLM‑based systems remain far from satisfying the demands of real‑world data analytics. DataGovBench provides a challenging benchmark for advancing research on LLMs capable of both answering analytical queries and discovering insights from data. Code and sample data are available at https://github.com/SoHasegawa/datagovbench.
Authors:Glen Pouliquen, Joseph Chazalon, Guillaume Chiron, Thierry Géraud, Ahmad Montaser Awal
Abstract:
This paper addresses the remote verification of the authenticity of Optically Variable Devices (commonly known as holograms) on identity documents. Typically placed over the cardholder's photo, these devices provide strong and easily verifiable security for human inspection but pose challenges for automated verification. Existing approaches easily cover static frauds (e.g. paper photocopy) and can be evaluated for such, but their capacity to detect real, dynamic fraud cases (e.g. handcrafted hologram) has not been evaluated to date because of the lack of public datasets. Furthermore, they are usually trained to detect known attack types, and few of them can generalize to new, unseen attacks. This work features three contributions to address these limitations: 1) a new public dataset, MIDV‑DynAttack, which extends the existing MIDV‑Holo dataset with realistic, static and dynamic attacks against identity document specimens, tripling the number of attack samples compared to the original dataset, 2) a novel verification method which can assess the authenticity of a specific hologram thanks to the analysis of its dynamic behavior and appearance, can be trained without dynamic attack samples, and exhibits new state‑of‑the‑art performance, 3) a benchmark of existing approaches which follows a clear evaluation protocol and emphasizes the inability of other approaches to deal with dynamic attacks, as well as new challenging attacks to deal with. Code and dataset are publicly available at https://github.com/EPITAResearchLab/pouliquen.25.icdar.
Authors:Sihang Nie, Jinxin Ji, Xiaofen Xing, Deyi Tuo, Chengbin Jin, Jialong Mai, Xiangmin Xu
Abstract:
While recent Large Language Model (LLM)‑based Text‑to‑Speech (TTS) systems have achieved remarkable naturalness, they predominantly rely on implicit end‑to‑end generation paradigms, resulting in coarse‑grained control. In scenarios demanding precise stylistic interventions and strict temporal alignment, such as audiobook narration and video dubbing, the inability to explicitly manipulate word‑level acoustic attributes remains a critical bottleneck. This limitation is primarily amplified by the severe scarcity of fine‑grained annotated datasets and the architectural challenge of integrating multi‑dimensional control signals into discrete autoregressive generation. To address this, we propose a unified framework for highly precise word‑level control. First, we construct WordVoice‑5A, a massive 4.7k‑hour bilingual dataset featuring five‑dimensional word‑level annotations (duration, boundary, energy, pitch and tone) developed through a rigorous linguistically‑guided pipeline. Second, we introduce WordVoice to transform the implicit generation process into an explicit, highly controllable paradigm. Specifically, we introduce a bound‑token mechanism within the LLM to formulate an explicit ``acoustic planning'' process, enabling adaptive multi‑task prosodic planning and flexible manual intervention. Furthermore, we augment the token‑to‑waveform stage with a fine‑grained acoustic modulation module, bridging the resolution gap to strictly align word‑level attributes between highly compressed discrete tokens and continuous waveforms. Extensive experiments demonstrate that WordVoice achieves superior, decoupled control over multiple acoustic dimensions while maintaining competitive zero‑shot synthesis stability. The code and audio samples are publicly available at https://xxh333.github.io/wordvoice‑demo/.
Authors:Ritabrata Chakraborty, Divy Kala, Nisheeth Bhooshan Gupta, Ganji Sreeram, Pailla Balakrishna Reddy, Makarand Tapaswi
Abstract:
Audio Descriptions (ADs) narrate visual content for Blind and Low Vision (BLV) audiences during gaps in audiovisual media. There is growing momentum around ADs in movies and TV shows, and with mandates from India's Central Board of Film Certification (CBFC), there is a need to expand ADs beyond English. Yet, there is no work that generates ADs for any Indian language. To address this gap, we present the first systematic study of ADs in Hindi, contributing to aspects such as data, generation, and evaluation. We introduce Andha‑Dhun, the first dataset of human‑authored Hindi ADs collected from 8 full‑length movies. We explore two approaches for generating ADs in Hindi: (i) directly from English dense video descriptions, and (ii) translating English ADs into Hindi. We evaluate these approaches using perplexity and LLM‑as‑a‑judge metrics to assess fluency and quality respectively. We also analyze movies that have both English and Hindi human‑authored ADs and find that naive translation introduces artifacts and narrows diversity compared to original Hindi ADs. Direct machine translation fails to adapt cultural references, while human‑translated ADs do better but still fall short. Our findings emphasize that the purpose of Hindi ADs is accessibility for Indian BLV audiences, and that this requires adapting content for the audience more than strict fidelity to the source.
Authors:Jihao Liu, Guoxiong Gao, Zeming Sun, Bin Wu, Shurui Liu, Jiedong Jiang, Haocheng Ju, Leheng Chen, Ronnie Cheng, Xiping Zhang, Bin Dong
Abstract:
Recent LLM‑based mathematical reasoning agents have begun to tackle research‑level problems and, in several cases, have contributed to the resolution of open problems. However, scaling and orchestrating such agents effectively remains challenging, due to the difficulty of coordinating parallel proof search while keeping intermediate claims organized and reliable. In this paper, we propose Danus, an orchestration system for research‑level mathematical reasoning centered on a shared fact graph as a global memory‑management mechanism. Danus consists of a main agent that performs planning and coordination, multiple worker agents that carry out proof search in parallel, and a stateless verifier that checks proposed mathematical claims before they are admitted into the fact graph. Each verified fact is stored together with its proof and logical dependencies, allowing the system to build long arguments incrementally while keeping the shared proof state organized. The main agent periodically summarizes the evolving proof state, redirects workers across promising directions, and supports interaction with human mathematicians through progress reports. We evaluate Danus through six research‑level case studies in algebraic geometry, singularity theory, and combinatorics, illustrating how the fact‑graph memory mechanism enables Danus to construct long, detailed mathematical proofs. Our results suggest that fact‑graph‑based orchestration provides an effective route toward scaling mathematical reasoning agents for long‑horizon research problems. Danus is open source at https://github.com/frenzymath/Danus.
Authors:Changti Wu, Bin Yu, Zhaolong Shen, Shijie Lian, Xiaopeng Lin, Cong Huang, Zhirui Zhang, Lei Zhang, Kai Chen
Abstract:
Vision‑Language‑Action (VLA) models are typically trained by imitation learning on large‑scale robot demonstration datasets, but more data does not necessarily yield better policies due to redundancy, noise, and uneven coverage. Existing data selection methods often assess demonstrations at either the trajectory or state‑action level, missing the reusable structures that compose long‑horizon behaviors. In this paper, we propose SIEVE, a structure‑aware data selection method for VLA imitation learning. SIEVE views demonstrations as compositions of reusable primitives and transition interfaces. It first discovers visuo‑motor primitives from segmented trajectories, then allocates selection budgets to composition patterns by maximizing reuse‑aware structural exposure under diminishing returns. Finally, it selects medoid trajectories within each composition‑pattern bucket to retain central, stable, and imitation‑friendly demonstrations. Experiments across multiple datasets, benchmarks, and VLA models show that SIEVE consistently outperforms competitive data selection baselines. Notably, SIEVE can surpass full‑data training while using only 50% of demonstrations and 50% of training steps, suggesting that reusable structure, captured through primitives and transitions, is an important signal for efficient VLA imitation learning.
Authors:Yuhang Wu, Shuxiang Zhang, Wee Hian Ching, Chi Zhang, Miao Liu
Abstract:
Recent text‑to‑image models such as DALLE‑3 excel at following diverse prompts yet remain blind to individual aesthetic preferences. We study personalized image generation, where models must align outputs with a user's implicit visual preferences based on a few historically preferred images and a short prompt. To this end, we introduce PIPBench, the first profile‑inclusive benchmark for evaluating personalized image generation. We further propose a novel data construction pipeline that leverages psychological and demographic profiling dimensions for both real‑user data collection and scalable agent‑based data generation. Using PIPBench, we conduct a thorough evaluation of representative line of methods. Our experiments reveal key limitations in existing methods, suggesting new challenges and opportunities for personalized text‑to‑image synthesis. Project page: https://wuyuhang05.github.io/PIPBench/
Authors:Sofiane Daimellah, Sylvie Le Hégarat-Mascle, Clotilde Boust
Abstract:
X‑ray fluorescence (XRF) spectroscopy is a key modality for material analysis in cultural heritage. However, automated learning from XRF spectra remains challenging: XRF spectra are complex one‑dimensional signals composed of sharp elemental peaks, broader structures, and background variations that are not taken into account by existing learning‑based models. This paper introduces XRFormer, a transformer architecture tailored to XRF spectra through a multiscale convolutional tokenizer that injects locality and multi‑resolution inductive biases before global self‑attention. The tokenizer progressively reduces spectral resolution while increasing embedding dimensionality, and the resulting token sequence is processed by a standard transformer encoder. We further investigate self‑supervised pretraining for XRF representation learning using Masked Spectral modeling (MSM) and a physics‑informed Peak Presence Prediction (PPP) objective. Experiments on the Pigments Checker STANDARD v.5 dataset for pigment identification and unmixing show that XRFormer consistently outperforms ViT, SpectralFormer (with and without CAF), and a 1D‑CNN baseline for pigment identification. For pigment unmixing, XRFormer achieves robust abundance estimation while maintaining significantly higher parameter efficiency than SpectralFormer, operating at a lower token resolution (128 vs. 512 tokens) and with less than half the number of parameters (1.5M vs. 3.37M). MSM yields consistent gains across both tasks, while PPP further enhances performance for both identification and unmixing when tuned with an appropriate peak prominence. These results highlight multiscale, modality‑aware tokenization as an effective and parameter efficient foundation for transformer‑based XRF modeling under data‑limited conditions. A GitHub repository is provided at https://github.com/sofiane1010/XRFormer.
Authors:Jinhong Deng, Limeng Qiao, Guanglu Wan
Abstract:
Visual counting is a fundamental pillar of multimodal intelligence, requiring a seamless integration of fine‑grained grounding and spatial reasoning. While Multimodal Large Language Models (MLLMs) have achieved remarkable success in qualitative scene understanding, their quantitative precision remains a significant bottleneck, often characterized by persistent numerical hallucinations. Existing counting benchmarks primarily focus on basic perception in simplified contexts, failing to capture the complex failure modes that emerge under logical constraints or adversarial conditions. To address these limitations, we introduce HoloCount, a holistic and diagnostically rich benchmark structured around a three‑level hierarchical taxonomy. HoloCount evaluates MLLMs across: (1) Semantic Counting, focusing on atomic and property‑based enumeration; (2) Analytical Counting, assessing logical composition through spatial and set‑based reasoning; and (3) Robustness Testing, probing model integrity against adverse scenarios and grounded counter‑priors, such as high‑density scenes and linguistic biases. Through an exhaustive evaluation of over 20 state‑of‑the‑art MLLMs, we reveal a critical performance gap: even top‑tier models degrade significantly as tasks transition from perception to complex analytical reasoning and adverse scenarios. Our findings provide a systematic landscape of current MLLM counting capabilities and offer a roadmap for developing more grounded and reliable multimodal systems. The dataset is available at https://mm‑mvr.github.io/HoloCount/.
Authors:Evgeny Shilov
Abstract:
Developers increasingly delegate real maintenance work to product‑grade coding agents, and many state tasks in their native language, in the style of a customer request rather than a curated English issue. Existing repository‑level agentic benchmarks do not measure this setting: their task statements are English by design. We introduce RuBench 1.0, a benchmark of 25 tasks mined from recent fix commits in five live open‑source repositories (aiohttp, aiogram, Laravel, NestJS, Fastify; Python, PHP, TypeScript, JavaScript), where each task is specified natively in Russian ‑‑ written from scratch in the style of an actual customer request, not translated ‑‑ and judged by the upstream maintainer's regression tests, which we withhold from release. All 25 fix commits postdate the training‑data cutoffs of every evaluated model, giving a contamination argument that holds task‑by‑task. We evaluate deployed product configurations (CLI agent + model + reasoning effort) ‑‑ Claude Code with Opus 4.8, Sonnet 5, and Haiku 4.5, and Codex CLI with GPT‑5.5 ‑‑ with three independent runs each, reporting pass@1 with task‑level confidence intervals, paired comparisons, dollar cost, and token usage. The best configuration resolves 78.7% of tasks; at N=25 only the gaps to the weakest model are statistically resolvable, which we state explicitly. Auditing full trajectories of a fifth, hors‑concours configuration (Claude Code + Fable 5, July 2, 2026 release), we caught the product silently substituting the model: on 5 of 25 tasks (20%) an official safeguard fallback re‑routed routine HTTP‑protocol fixes to Opus 4.8 ‑‑ direct, reproducible evidence that the deployed product, not the model, is the unit actually measured. We release task statements, metadata, full agent trajectories, and diffs; grading oracles are withheld, with a SHA‑256 manifest committed at publication time.
Authors:Glen Pouliquen, Joseph Chazalon, Guillaume Chiron, Oriol Ramos Terrades, Thierry Géraud, Ahmad Montaser Awal
Abstract:
Robust remote verification of identity documents relies on analyzing faint, transparent security features like Optically Variable Devices (OVDs), or "holograms", within user‑captured videos under uncontrolled conditions. Current systems, however, face critical limitations: existing methods often treat video frames in isolation, neglecting the intrinsic dynamic nature of OVDs and leaving systems vulnerable to swapping attacks, or focus on general holographic presence and lack the ability to verify specific OVD types. Moreover, the economic infeasibility of frame‑by‑frame video annotation makes supervised training impractical. In this work, we introduce two novel approaches for verifying the dynamic behavior of transparent OVDs protecting the holder's portrait, specifically designed for open‑set scenarios where attack types are unknown during training. We demonstrate that these approaches can be trained without any attack samples in a self‑supervised setting, surpassing previous state‑of‑the‑art methods on public datasets while adhering strictly to industrial constraints. Our results confirm that modeling temporal dynamics is essential for defeating sophisticated attacks under realistic conditions, and underscores the promise of sequence modeling and anomaly detection for OVD verification. Code is available at https://github.com/EPITAResearchLab/pouliquen.26.icdar.
Authors:Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu, Yunnan Wang, Yibin Yan, Yong Wang, Shuailei Ma, Xinyang Wang, Yibin Liu, Shuai Yang, Tianxiang Zhou, Kejia Zhang, Lei Zhou, Cheng Su, Nan Xue, Bin Tan, Han Zhang, Youchao Zhang, Fei Liao, Xing Zhu, Yujun Shen, Kecheng Zheng
Abstract:
Despite recent progress of VLA foundation models, the disparity between laboratory conditions and real‑world applications continues to impede their practical implementation. To bridge this gap, we present LingBot‑VLA 2.0, which advances LingBot‑VLA through improvements in three functional domains. (1) Generalization across tasks and embodiments. Compared to the previous version, we revamp the data processing pipeline and curate around 60,000 hours of data for pretraining, including 50,000 hours of robot trajectories spanning 20 robot configurations and 10,000 hours of egocentric human videos. (2) Expanded action space in addition to dual‑arm hardware platforms. In particular, our system accommodates degrees of freedom for the heads, waists, mobile bases, and dexterous hands, thereby empowering the robots to tackle more complex tasks in practical scenarios. (3) Predictive dynamics modeling for improved temporal reasoning. Specifically, we formulate future prediction as a proxy task, facilitated by a video representation model for semantic priors and a depth estimation model for geometric cues. Evaluations on the GM‑100 benchmark, conducted in a generalist setting, validate the beneficial impact of these proposed modifications. Furthermore, benefiting from the expanded pretraining data that covers whole‑body degrees of freedom, LingBot‑VLA‑2.0 demonstrates strong cross‑embodiment long‑horizon mobile manipulation capability across the two robotic platforms.
Authors:Jiazi Wang, Nonghai Zhang, Qiushi Xie, Zeyu Zhang, Yufeng Chen, Yang Zhao, Ling Shao, Hao Tang
Abstract:
Vision‑language models (VLMs) have made interactive digital museums increasingly feasible by connecting 3D digitization with natural‑language artifact exploration. However, in cultural heritage domains such as ancient Greek pottery, reliable VLM assistance is limited by two challenges. First, open‑ended interpretation requires grounding fine‑grained 2D/3D visual evidence in specialized curatorial knowledge, yet the retrieval process may introduce weak sources and unverifiable references. Second, when the available evidence is incomplete, noisy, or ambiguous, VLMs often produce confident but unsupported answers instead of calibrated uncertainty. To address these challenges, we propose VaseMuseum, a lightweight and modular multimodal agent framework for intelligent digital museums of ancient Greek pottery. VaseMuseum combines an interactive virtual museum with VaseAgent, which supports both 2D images and 3D artifacts through multimodal perception, 3D‑aware reasoning, external knowledge retrieval, and inference‑time reliability control. Specifically, VaseAgent retrieves evidence from authoritative web and museum knowledge sources, and source‑level control selects diverse and verifiable evidence before generation. Meanwhile, response‑level control checks generated claims against the evidence pool and encourages neutral, evidence‑bounded answers when support is insufficient or conflicting. Moreover, a training‑free GRPO‑style selection mechanism favors responses with valid references and calibrated confidence without updating the VLM backbone. Experiments in a realistic digital museum simulation show that VaseMuseum improves citation validity, reduces hallucinations on knowledge‑intensive queries, and produces more neutral answers under ambiguity compared with search‑enabled VLM baselines.
Authors:Alexander Demin
Abstract:
A standard way to control expression swell in computer algebra is to use multi‑modular or evaluation‑interpolation methods. In computations involving Gröbner bases, these techniques typically require repeatedly computing Gröbner bases of specializations of the same ideal. These repeated computations can be accelerated through precomputation, notably using Traverso's tracing. We present Groebner.jl (https://github.com/sumiya11/Groebner.jl), a Julia implementation of the F4 algorithm that exposes Traverso's tracing through a reusable public interface. The implementation supports SIMD‑friendly coefficient types, such as tuples of machine integers, which Julia compiles to efficient code with little manual intervention. This lets other Julia software leverage tracing to obtain speedups in applications such as structural identifiability of ordinary differential equation models and polynomial system solving.
Authors:Zhen Li, Gang Cao, Tian Zhang, Lifang Yu, Shaowei Weng
Abstract:
The rapid advancement of large‑scale generative models has accelerated the spread of highly deceptive AI‑generated images, making generalized synthetic image detection a critical imperative. Existing forensic networks often struggle with cross‑model generalization and realworld degradations due to their reliance on single‑domain representations and conventional binary classification optimization. To overcome these limitations, we propose RNSIDNet, a novel forensic framework that achieves robust detection through enhanced RGB‑Noise representation learning. Specifically, our method employs a dual‑branch architecture where global RGB semantics, extracted by an attention‑refined CLIP backbone, dynamically modulate highfrequency noise artifacts captured by Bayar convolutions via a Feature‑wise Linear Modulation (FiLM) module. To further enhance the learned representations, we design a Hard Sample‑aware Contrastive Learning (HSCL) strategy. By explicitly penalizing challenging training samples, HSCL reshapes the latent feature space to maximize the discriminative margin between pristine and synthetic domains. Extensive experiments across eight public benchmark datasets verify that our model achieves state‑of‑the‑art performance, delivering superior generalization ability, robustness, and computational efficiency. Code and dataset will be publicly available on https://github.com/multimediaFor/RNSIDNet.
Authors:Sharayu N. Deshmukh, Md Rashidunnabi, Nelton Tiago Gemo, Kurundkar G. D., Mahamune M. R., Nilesh K. Deshmukh
Abstract:
Deepfake image detection is currently served by three fundamentally different paradigms: commercial APIs, zero‑shot vision‑language models (LLMs), and open‑source detectors. Despite their widespread use, these paradigms are rarely evaluated under a common protocol, making direct comparison difficult. We introduce VendorBench‑100, a cross‑paradigm benchmark that evaluates 36 representative models using a single adversarial 100‑image corpus, a unified output schema, and a common evaluation framework. To ensure reliable assessment under the corpus's intentional class imbalance, models are ranked primarily by the Matthews correlation coefficient (MCC), with ROC‑AUC reported as a threshold‑independent measure of ranking ability. Rather than maximizing dataset size, VendorBench‑100 emphasizes challenging real‑world scenarios through a curated taxonomy of eight edge‑case families, including face swaps, text‑to‑video stills, AI photo edits, avatar compositing, opaque‑provenance images, and compressed research frames. Our evaluation shows that commercial APIs achieve the strongest median performance, followed by vision LLMs and open‑source detectors. However, individual open‑source models remain competitive with the best vision LLMs. More importantly, we identify a consistent divergence between ranking ability (ROC‑AUC) and operating‑point quality (MCC), demonstrating that strong score discrimination does not necessarily produce reliable default‑threshold decisions. This metric disagreement, rather than any single leaderboard ranking, is the central finding of the benchmark. We release the complete evaluation framework and benchmark results to support reproducible future research. The source code and data are available at: https://github.com/sharayu‑20/vendorbench‑100
Authors:Lihua Wei, Huatong Gao, Jia Gong, Zhiyu Tan, Hao Li, Jun Liu, Zhihua Ren
Abstract:
Magnetic resonance imaging (MRI) super‑resolution is vital for improving diagnostic accessibility, yet most methods treat it as a deterministic mapping from a fixed low‑resolution input to a high‑resolution target. This overlooks a key property of MRI acquisition physics: spatial resolution and signal‑to‑noise ratio (SNR) are inherently coupled, making any given low‑resolution scan merely one of many possible realizations under varying acquisition trade‑offs. We rethink MRI super‑resolution as a physics‑aware reconstruction problem, in which the goal is to identify the optimal resolution‑SNR configuration and then super‑resolve it to obtain high‑quality MRI results. A key implication of this formulation is that MRI resolution becomes dynamic rather than fixed. To handle such resolution‑heterogeneous inputs, we adapt 2D Gaussian Splatting (2D GS) to MRI by formulating reconstruction as a coordinate‑based, resolution‑agnostic rendering problem. To further enhance fidelity, we introduce three innovations: (1) a prior‑aware Gaussian representation that combines an Anatomical Structure Prior for tissue‑specific kernel initialization with an Imaging System Prior that captures hardware characteristics via a covariance dictionary; (2) a physics‑constrained signal modeling scheme that predicts intrinsic tissue parameters (proton density rho and effective relaxation rate R2) and synthesizes intensities through governing physical equations, ensuring biophysically plausible contrast; and (3) a meta‑learning framework that alleviates paired‑data scarcity by pretraining on simulated data and adapting to real‑world conditions. Extensive experiments on dynamic‑resolution datasets and standard benchmarks demonstrate that our method achieves state‑of‑the‑art performance, highlighting its strong potential for clinical deployment.
Authors:Tianyang Liu, Canwen Xu, Fangyu Lei, Nikki Lijing Kuang, Jixuan Chen, Tao Yu, Julian McAuley, Zhewei Yao, Yuxiong He
Abstract:
Major cloud data platforms now expose large language model capabilities as native SQL functions, enabling analysts to perform classification, filtering, sentiment analysis, extraction, similarity search, and aggregation within ordinary SQL queries. Yet existing text‑to‑SQL benchmarks evaluate only conventional SQL and provide no signal on whether models can generate such AI‑native SQL. We introduce Spider 2.0‑AIFunc, a benchmark of 465 verified instances across 125 real‑world databases covering six types of AI functions on the Snowflake platform. Starting from an existing enterprise text‑to‑SQL benchmark, we construct Spider 2.0‑AIFunc through an agent‑based pipeline that rewrites source tasks into AI‑native form, simultaneously transforming target queries and refining natural language instructions to make the intended AI‑native solution explicit and reduce ambiguity. All instances pass a multi‑round repeated execution protocol across temporally separated windows to confirm result stability before release. Evaluating ten state‑of‑the‑art language models, we find that the strongest proprietary models reach 67‑70% execution accuracy while the best open‑source model achieves 58.1%, a gap driven primarily by errors in predicate specification, schema grounding, and AI function parameterization. Agent frameworks designed for traditional text‑to‑SQL challenges, such as schema retrieval and relevant table selection, do not transfer effectively to AI‑native SQL: a minimal agent setup consistently matches or outperforms more elaborate alternatives, suggesting that the strategies these frameworks employ are less critical in this setting. Data are available at https://github.com/Leolty/Spider2‑AIFunc .
Authors:Onur Eker, Erkut Erdem, Aykut Erdem
Abstract:
Enhancing videos under extreme low‑light conditions remains challenging due to the difficulty of balancing restoration quality and computational efficiency in resource‑constrained settings. This paper introduces EeveeDark, a low‑light video enhancement framework that combines the spatial richness of sensor‑level RAW data with the temporal precision of event streams. Central to our model is a Binary Neural Network (BNN) architecture that reduces computational overhead by quantizing weights and activations while preserving detail. EeveeDark incorporates (i) modality‑specific binary encoders for processing RAW frames and event data, (ii) a lightweight fusion block for integrating spatial and temporal cues, and (iii) an event‑guided skip gating mechanism for dynamic spatiotemporal refinement. Experiments on synthetic and real‑world datasets show that EeveeDark outperforms prior BNN‑based methods and offers a favorable performance‑efficiency trade‑off compared to full‑precision models. The project page is available at https://cyberiada.github.io/EeveeDark.
Authors:Team Moxin, Deyi Ji, Tianrun Chen, Xin Zhang, Jiale Yang, Qi Zhu, An Zhao, Zihao Xie, Han Wang, Xuanyi Liu, Yixiang Zhou, Pei Liu, Yi Tan, Cheng Chen, Dayi Zhu, Mingyu Wei, Hanjie Xu, Jun Liao, Siqi Li, Lingyu Lu, Hongye Fang, Hongming Tan, Youjiang Zhu, Taiyu Zhang, Zejian Li, Chaotao Ding, Lanyun Zhu, Yunhe Pan, Lingyun Sun
Abstract:
The future of World Models depends not only on scaling model capability, but also on scaling practicality and inference efficiency. High‑frame‑rate inference enables responsive perception, planning, and control in real‑world autonomous systems. To this end, we present MoWorld, a cost‑effective yet high‑performance Flash World Model with an end‑to‑end framework spanning data generation, pre‑training, distillation, and efficient inference, enabling up to 50 FPS real‑time interaction with cinematic visual quality without the need of high‑end GPUs. To enable large‑scale real‑world deployment, MoWorld jointly optimizes model capability and cost throughout the entire development pipeline. Specifically, unlike existing approaches that primarily rely on large‑scale video corpora, MoWorld is built upon a scalable 3D‑native data engine accumulated from our large‑scale 3D vision and generative modeling pipeline, enabling the efficient construction of geometrically consistent training data across diverse real‑world and synthetic environments. Based on this foundation, a curriculum cross‑frame pre‑training strategy for stable and scalable World Model learning, an efficient denoising‑step distillation algorithm to reduce diffusion training cost, and a mixed‑precision parallel inference framework for low‑cost real‑time deployment. MoWorld is the first real‑time interactive World Model built on the Neural Processing Unit (NPU) and can achieves up to 50 FPS in such the devices, enabling practical and efficient deployment at scale. Comprehensive evaluations demonstrate that MoWorld achieves leading performance; notably, its average inference cost is only 30%‑50% of that of existing World Models, providing a practical foundation for large‑scale real‑world applications of World Models. We also demonstrate diverse applications of MoWorld.
Authors:Hong Lyu, Mingru Yang, Qianhua He, Yanxiong Li, Jinxin Huang, Zhengyu Pei
Abstract:
There are some datasets of varying scales for audio classification (AC) applied to different tasks. However, annotated data is limited for most scenarios, such as domestic environments. To address this challenge, we propose an Automatic Audio Annotation Pipeline‑‑TriA Pipeline, which can efficiently convert audio from various scenarios into high‑quality training data with audio event annotations. A TriA dataset was constructed with the TriA Pipeline, over 2130 hours of audio covering 431 audio classes. Furthermore, we partitioned a prior‑knowledge‑guided subset (TriA_\mathrmGK) from TriA and conduct comparative experiments on three domestic AC tasks. Comparing the result on manually annotated data only and that on manually annotated data combines TriA_\mathrmGK, TriA_\mathrmGK could achieve average relative gains of 3.97% in accuracy and 3.35% in Macro‑F1, validating the effectiveness of TriA_\mathrmGK and the TriA Pipeline.
Authors:Alexander Rombach, Chantale Lauer, Nijat Mehdiyev
Abstract:
Large language models (LLMs) can generate BPMN process models from natural‑language descriptions, yet supervised fine‑tuning (SFT) limits their output quality to the patterns present in the training data. Reinforcement learning (RL) can optimize beyond this ceiling using external quality measures, but how the reward function should be designed when quality is multi‑dimensional remains unexplored. We present a systematic investigation of reward function design for RL‑based process model generation, training two LLM families (Llama~3.1 8B, Qwen~2.5 14B) under 48 configurations using Group Sequence Policy Optimization with rewards derived from an automated evaluation framework comprising 38 metrics across syntactic, pragmatic, and semantic quality. Three findings emerge. First, RL significantly improves pragmatic and syntactic quality while preserving semantic fidelity, reducing output variability by more than sixfold. Second, equal reward weighting consistently outperforms targeted weighting: emphasizing a specific dimension fails to improve it and can collapse the model into a low‑quality mode. Third, design choices interact with model architecture in non‑trivial ways: the invalidity penalty is essential for one model but irrelevant for the other, and SFT initialization is indispensable for one architecture but counterproductive for another. These results demonstrate that reward composition is a primary determinant of optimization outcomes, with effects as large as the decision to apply RL itself. The findings generalize to any structured generation task where quality is assessed along multiple automated dimensions. We release our implementation and experimental code at https://github.com/chlauer99/RL_for_process_modeling.
Authors:Mohsen Ghafoorian, Denis Korzhenkov, Adil Karjauv, Ioannis Lelekas, Noor Fathima, Spyridon Stasis, Hanno Ackermann, Boris van Breugel, Markus Nagel, Fatih Porikli, Animesh Karnewar, Amirhossein Habibian
Abstract:
Recent advances in video diffusion have been driven by scaling transformer‑based architectures to billions of parameters, substantially improving visual fidelity and motion coherence. In contrast, existing mobile video diffusion models remain limited to relatively small parameter budgets, typically 0.4‑1.8B, restricting generation quality. In this work, we show that high‑quality mobile video generation does not require small models. Instead, we demonstrate that a server‑scale 5B‑parameter video diffusion transformer can be deployed efficiently on memory‑constrained mobile hardware through recurrent reformulation and structured compression. Starting from Wan2.2‑5B, we rely on a recurrence distillation framework that converts video generation into a chunk‑wise autoregressive process with constant‑memory attention computation. Combined with causal linear attention, the model operates as an RNN at inference time while preserving temporal coherence across chunks. We further propose a learnable attention head pruning method based on binary per‑head gates optimized end‑to‑end using a noise‑biased sparsity objective and distillation‑based finetuning. Together with sampling‑step distillation and memory‑optimized VAE decoding, MobileWan becomes the first 5B‑scale video diffusion model deployable on a commercial mobile device. Our system generates 5‑second 480x832 videos at 16 FPS in 20 seconds end‑to‑end latency, achieving a VBench score of 83.79 and establishing a new state of the art in mobile video generation. Project page: https://qualcomm‑ai‑research.github.io/mobilewan
Authors:Chenxu Wang, Yongkun Yang, Boyuan Du, Shiwei Lin, Huaping Liu
Abstract:
Deliberation plays a crucial role in collaboration; when humans work together, they naturally engage in communication to align information and reach an agreement. In this paper, we investigate deliberative large language model (LLM) agents under partially observable joint decision‑making tasks. We formalize deliberative collaboration as a cooperative joint decision problem with partial and asymmetric observations, and introduce a scalable benchmark that instantiates this problem across multiple task settings and domains in which agents must exchange information through deliberation to reach a joint decision with a shared reward. We then instantiate a reference scaffold and evaluation protocol for deliberative agents and conduct a systematic evaluation of a range of representative LLMs. The results reveal that complex deliberative collaboration tasks continue to challenge state‑of‑the‑art language models. Even with the aid of external mathematical tools, language models may fail in either the deliberation process for aligning information or the complex reasoning process for making the decision. On the other hand, diagnostic analysis reveals that the deliberation process may also provide opportunities for reflection and error correction, sometimes improving performance over centralized baselines. Altogether, our work establishes a foundation for evaluating and improving LLM agents in deliberative collaboration and provides insights into the strengths, limitations, and properties of current LLM‑based multi‑agent systems.
Authors:Wanglong Lu, Lingming Su, Kaijie Shi, Minglun Gong, Xiaogang Jin, Hanli Zhao, Xianta Jiang
Abstract:
Recent diffusion‑based generative models have shown impressive performance in image generation and editing. However, due to memory limitations and the high cost of collecting high‑resolution training images, existing methods are typically restricted to inputs with linear resolutions below 1K. In contrast, photos captured by modern mobile devices often reach linear resolutions up to 8K, revealing a significant gap between current capabilities and real‑world demands. Simply upscaling low‑resolution edited results often results in visually enlarged but blurry images that lack fine details. This paper introduces UltraDiffEdit, a novel, tuning‑free image editing framework that extends off‑the‑shelf latent diffusion models (LDMs) to ultrahigh resolutions. UltraDiffEdit employs a multi‑scale progressive editing strategy, iteratively blending high‑resolution edited content with unedited areas in a coarse‑to‑fine manner. We employ multi‑patch encoding to preserve both edited and unedited visual details within the latent space. To mitigate editing artifacts, our global‑local consistency denoising technique consistently integrates edited and unedited latent features, ensuring smooth transition at editing boundaries from the latent representation to the final image. We also introduce a patch‑based hybrid sampling approach that captures local, intermediate, and global features, ensuring semantic coherence and enhancing fine detail during denoising. We conduct extensive experiments demonstrating UltraDiffEdit's superior editing quality and flexibility: it can handle image resolutions up to 8K using only a single NVIDIA GeForce RTX 3090 GPU. The source code is publicly available at https://github.com/LonglongaaaGo/UltraDiffEdit.
Authors:Woo Jae Kim, Kyle Min, Suhyeon Ha, Joonsung Jeon, Sung-eui Yoon
Abstract:
Multi‑perturbation adversarial training (MAT) aims to achieve robustness against multiple \ell_p perturbations but suffers from robustness trade‑offs between different threats. To address this, we employ a mixture of experts (MoE) to route different threats through distinct model pathways. However, naive application of MoE encounters two critical challenges: experts tend to overlook threat‑specific features and redundantly capture features shared across threats, and gating networks suffer from threat‑agnostic routing where they learn nearly identical routing patterns across threats, thus preventing the construction of threat‑specific model pathways. To this end, we propose Robust Mixture of Low‑Rank Experts (RoME), where each expert is a low‑rank additive update to the shared backbone, allowing it to capture threat‑common features while experts focus on threat‑specific information. To address threat‑agnostic routing, RoME introduces (i) dual‑scale gating that exploits threat‑discriminative signals from local and global level features, and (ii) threat‑guided gating diversification that enforces diverse expert utilization across threats. Extensive experiments demonstrate that RoME outperforms existing state‑of‑the‑art MAT in union robustness and natural accuracy and improves robustness against unseen threats. Codes are available at https://github.com/wkim97/RoME.
Authors:Zheng Guo, Jiaqi Cui, Haocheng Xiong, Jize Han, Bo Liu, Qianwen Zhang, Rui Chen, Yan Wang
Abstract:
Non‑invasive prediction of Gleason Grade Group (GGG) in prostate cancer using multiparametric MRI (mpMRI) is clinically vital for reducing unnecessary biopsies. Existing GGG prediction methods face two major limitations. First, they often overlook non‑image information critical for GGG prediction, including age, prostate‑specific antigen (PSA), and expert priors embedded in radiology reports. Second, they tend to oversimplify GGG as flat categorical labels, failing to account for its intrinsic hierarchy of primary and secondary Gleason patterns. To this end, we propose a novel Knowledge‑Driven Ordinal‑Aware Learning (KOAL) framework with three synergistic modules. Specifically, the Clinical‑Context Modulation (CCM) module uses clinical variables (e.g., age and PSA) to dynamically modulate discriminative image representations. The Knowledge‑Guided Prototype Alignment (KGPA) module leverages an LLM to extract group‑specific expert knowledge from training radiology reports and clinical guidelines, producing offline semantic anchors describing grade‑specific radiological findings without requiring patient‑specific reports at inference. Through prototype contrastive alignment, patient‑specific mpMRI representations are matched with these anchors to promote pathology‑aligned representation learning. The Hierarchical Ordinal‑aware Constraints (HOC) module decouples primary and secondary Gleason pattern prediction and maps their probabilistic outputs to GGG via a Differentiable Bio‑logic Mapping Layer (DBML), ensuring pathological grading consistency. Experiments on public PI‑CAI and in‑house datasets demonstrate that KOAL outperforms state‑of‑the‑art methods. Code is available at: https://github.com/Gother‑GZ/KOAL.
Authors:Hanan Gani, Tejal Kulkarni, Madhoolika Chodavarapu, Nicklas Hansen, Manmohan Chandraker
Abstract:
Pretrained video generative models are promising backbones for visuomotor control, but their imagined futures often drift from task intent and are not reliably action‑conditional. As a result, these models can be difficult to use for planning or policy extraction. To address these limitations, we propose RoboTALES, a single‑stage framework that learns task‑aligned simulated futures and uses them to train robot policies. Our approach introduces two key innovations: (1) a hierarchical LLM‑based planner that breaks complex tasks into a sequence of subgoals to guide the model's imagination; and (2) a VLM‑based critic that evaluates these ``imagined'' futures and uses reward‑based feedback to keep the model's internal representations focused on the goal. By anchoring the video generator in abstract reasoning, we produce temporally consistent rollouts and more coherent actions. We evaluate RoboTALES on diverse manipulation tasks from RoboCasa and LIBERO10, and show that our method consistently outperforms existing methods, especially in long‑horizon tasks. Our code and models are publicly available at https://github.com/hananshafi/RoboTALES.
Authors:Jun Wei, Xinchang Liu, Yu Liu, Chuhua Yang, Shuhui Wang, Hui Huang
Abstract:
Pixel‑level annotation remains a major bottleneck in medical image segmentation, making weak supervision an attractive yet under‑constrained alternative. We propose OBBSeg, an intermediate supervision paradigm guided by Oriented Bounding Boxes (OBBs) that bridges the gap between full and weak supervision. By jointly encoding spatial extent and orientation, OBBs provide compact geometric supervision that better aligns with elongated or anisotropic lesions, reducing the ambiguity of coarse box annotations. To mitigate the inherent rectangular bias of OBBs, we introduce a Mask‑to‑OBB loss, a differentiable formulation that enforces geometric consistency between predicted masks and OBB regions. Furthermore, we incorporate prompt‑driven semantic guidance through two complementary modules‑PAFE and DBFE‑which enhance foreground representation and suppress background interference. Extensive experiments on 13 datasets across 5 imaging modalities show that OBBSeg not only outperforms existing weakly supervised methods but also achieves performance comparable to fully supervised approaches, demonstrating its potential for efficient and scalable medical image segmentation. The code is available at https://github.com/StarLxc3/OBBSeg.
Authors:Shenbo Xie, Mingrui Cai, Xu Yang, Yifei Liu, Changxing Ding
Abstract:
Human‑Object Interaction (HOI) video generation aims to synthesize realistic videos of humans manipulating diverse objects, serving as a promising avenue for AI‑driven live streaming e‑commerce. A primary obstacle in this domain lies in the complexity of modeling fine‑grained physical dynamics and the intricate spatial‑temporal coordination between human hands and objects. Existing approaches to this problem typically rely on dense temporal guidance, e.g., frame‑wise hand‑object pose sequences, to strictly control the interaction process. However, such dense guidance incurs high annotation costs and affects motion synthesis diversity. To overcome these limitations, we introduce SparseCtrl‑HOI, a novel sparse temporal control framework for HOI video generation. It requires only a few keyframes that capture interaction states at designated timestamps. Specifically, we employ a Time‑Controlled Rotary Positional Embedding (TiRoPE) mechanism to temporally anchor these keyframes while preserving their spatial integrity. Subsequently, to govern the dynamics across intermediate frames, we propose a Motion Prior Injection Module that leverages Multimodal Large Language Models (MLLMs) to extract high‑level motion priors. This empowers the model to hallucinate logically and physically plausible transitions. Furthermore, we build SparseHOI‑5K, a high‑quality and richly annotated dataset for HOI video generation with sparse temporal control. Comprehensive evaluations confirm that our method substantially reduces annotation overhead while synthesizing superior live‑streaming e‑commerce videos. Both our code and dataset are publicly available at https://mpi‑lab.github.io/SparseCtrl‑HOI.
Authors:Zifan Wang, Siyu Chen, Wenzhuo Song
Abstract:
While signed social recommendation has shown great potential by modeling both trust and distrust relations, its effectiveness is often hindered by structural noise and data sparsity. In this work, we first identify a fundamental inconsistency across the structural, propagation, and semantic layers of existing models, which leads to biased representations learned from sparse or noisy datasets. Furthermore, we observe that most existing methods treat the observed graph as fixed, failing to bridge the gap between noisy topologies and reliable social semantics. To address these issues, we propose a unified framework named SSC‑Loop that treats signed social recommendation as the maximization of structural consistency. SSC‑Loop includes three dedicated modules: ESA‑DA for structural consistency, a P/N/O propagation mechanism for propagation consistency, and a contrastive learning objective for semantic consistency. Experiments on Epinions demonstrate that SSC‑Loop achieves strong performance on explicit signed social rating prediction, while auxiliary results on Slashdot under a derived link‑existence setting further suggest its ability to exploit signed social structures. Source code is available at https://github.com/Refrainwww/SSC‑Loop.
Authors:Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Jiapeng li, Qunzhong Wang, Tailai Chen, Qianshan Wei, Chuan Xiao, Shanyu Rong, Yangfu Li, Yanhan Zhou, Yunpu Ma, Yifan Zhang, Xiangyu Yue
Abstract:
Training multimodal search agents to perform multi‑hop reasoning remains challenging due to a fundamental structural disconnect: existing pipelines construct training data, search environments, and reward signals independently, causing synthesized structural metadata to be discarded, environments to rely on irreproducible external engines, and RL rewards to remain sparse at the trajectory level. We present SearchEyes, which uses a typed knowledge graph as the backbone of a \emphsimulated search world that unifies all three components. We propose Perception‑Knowledge Chains (PKC) to sample constrained multi‑hop paths over the visual‑knowledge intersection of Wikidata5M, retaining hop‑level entity metadata that simultaneously defines a self‑contained search world and step‑level reward anchors. We further propose Hop‑Anchored Policy Optimization (HaPO), which reuses these anchors for step‑level credit assignment without a separately trained process reward model. Experiments on six multimodal knowledge‑intensive benchmarks show that SearchEyes achieves state‑of‑the‑art performance among open‑source multimodal search agents, with SearchEyes‑27B improving over the strongest open‑source baseline by 6.2 points on average.%
Authors:Mingyi Shi, Xuelin Chen, Taku Komura
Abstract:
Synthesizing hand motion that matches the full body motion and the semantic labels is a difficult task due to their high degrees of freedom and the lack of semantic labels. To cope with this issue, we propose a prior‑first, condition‑second framework for body‑conditioned hand motion completion. Our framework first learns a generic body‑hand kinematic prior from large‑scale unstructured and unlabeled motion data, capturing the intrinsic coordination between global body dynamics and hand articulation. Semantic control is then introduced through lightweight adaptation on top of the frozen prior, avoiding the need to relearn kinematic structure for each control interface. Our framework centers on a streaming, autoregressive body‑hand prior that generates coherent, kinematically consistent hand motion from body dynamics in real time, using structured kinematic modeling to maintain mechanical body‑hand coupling. To enable practical controllability under limited supervision, we introduce semantically‑layered adapters that inject conditioning signals at appropriate kinematic levels, supporting both self‑supervised attribute control and weakly supervised text‑driven control with only a few hours of labeled data. Extensive evaluations demonstrate that our framework improves kinematic plausibility, robustness, and controllability compared to end‑to‑end conditioned baselines, particularly in low‑resource and cross‑dataset settings. We further showcase real‑time inference and an interactive authoring workflow, highlighting the applicability to production animation pipelines. Homepage: https://AIGAnimation.github.io/HandPrior/
Authors:Jean-Francois Bonbhel
Abstract:
We present K‑ABENA (K‑Adaptive Backpropagation with Error‑based N‑exclusion Algorithm), a selective gradient computation framework that reduces per‑iteration training cost by excluding a fraction of low‑loss ("minor") observations from the backward pass. Its canonical form (v3) combines a defensive‑mixture sampling design over the minor set with Horvitz‑Thompson inverse‑probability reweighting, yielding a design‑unbiased Horvitz‑Thompson gradient estimator (Lemma 2) and whose self‑normalized practical variant carries a bias of order O(1/m) with an explicit constant (Lemma 3). We prove an O(1/sqrt(T)) non‑convex convergence guarantee for SGD under the estimator, with an additive term that quantifies the residual bias (Theorem 1). We further prove that uncompensated loss‑based selection ‑ a family that includes OHEM, SBP, and the two earlier K‑ABENA variants ‑ admits no stationary point at any minimizer where its selection bias is bounded away from zero (Proposition 2), and we quantify this failure empirically: at 0.17% class imbalance, uncompensated variants reach test AUC 0.53‑0.62 versus 0.9998 for full‑batch SGD, while the compensated estimator attains 0.9991 at identical 28.4% compute savings. On real datasets (Breast Cancer, Digits, Wine, Diabetes) the compensated estimator is statistically indistinguishable from full‑batch SGD (paired permutation tests, p >= 0.5; Section 7) while saving 28‑54% of per‑epoch gradient computation. A biased "regularized mode" (the earlier half‑domain variant) is retained as an option with a proven exact bias decomposition (Lemma 5) and quantified contraindications: it collapses to 0.386 accuracy under 40% label noise (baseline: 0.832) and to 0.53 AUC under extreme imbalance. Every advantage and every limitation reported in this paper is either proved or measured; all experiments are CPU‑scale (NumPy/scikit‑learn) and their scope is stated explicitly.
Authors:Cemil-Andrei Dilmac, Florinel-Alin Croitoru, Radu Tudor Ionescu
Abstract:
Coreset selection aims to identify a small and highly representative subset of a massive dataset for efficient model training. The problem remains challenging even in the few‑shot knowledge distillation (KD) setup, where a full‑scale pre‑trained teacher informs the student network. Typical sample selection strategies often struggle to surpass the random selection baseline. In this paper, we showcase few‑medoids, an embarrassingly simple coreset selection strategy that chooses the samples closest to the centroid (average image) of each class. We present extensive KD experiments on four datasets, covering a wide range of image classification problems, and three teacher‑student model pairs, comprising both convolutional and transformer networks. Although the proposed method is embarrassingly simple, our empirical results indicate that few‑medoids is able to consistently surpass the random selection baseline, as well as the other coreset selection strategies. We therefore consider that few‑medoids can be used as a drop‑in replacement for commonly‑used baselines (e.g. herding or k‑center Greedy), in future research on coreset selection. To reproduce the reported results, we publicly release our code at https://github.com/CemilAndreiDilmac/Few‑Shot‑KD‑Coreset.
Authors:Sishun Liu, Sajal Halder, Ke Deng, Yan Wang, Xiuzhen Zhang
Abstract:
Information Operations on social media networks have been identified as a significant threat to democracy and modern society, but they are challenging and expensive to detect by humans. Existing supervised IO detection methods fail to capture the dynamic nature of evolving IO user behavior, while existing unsupervised approaches rely on oversimplified assumptions of coordination among IO users that may not exist in practice. To overcome the limitations of existing methods, we formulate IO user detection as an anomaly detection problem and propose a novel unsupervised IO user detection approach called Temporal‑bEhavior‑laNguage Signals for information Operation Recognition (TENSOR), which leverages multimodal data, including temporal online user behavior, such as message posting activities, and the textual content of the messages. The motivation is that IO users are typically a very small fraction of all online users and have unique temporal behavioral and language patterns. Specifically, we train a Temporal Point Process (TPP) to capture abnormal temporal behavioral patterns of IO users because they are known to behave in a coordinated manner for IO campaigns. We further introduce a novel evidence function that converts LLM responses, which are generated from user post timelines, into quantitative scores to adjust the TPP outputs for better IO user detection. Experimental results show that TENSOR outperforms the baselines on five real‑world IO datasets. Code is available at https://github.com/xiuzhenzhang/TENSOR.
Authors:Suraj Yadav, Anjaneya Sharma, Siddharth Yadav
Abstract:
Deep neural networks trained with Empirical Risk Minimization (ERM) often fail under distribution shifts because they exploit spurious correlations between object labels and background context. Recent generative approaches address this issue by creating counterfactual images with altered contexts, but typically use these samples as standard data augmentation, leaving the model free to retain background‑sensitive representations. We propose a two‑stage framework that uses generative intervention to explicitly learn background‑invariant visual representations. First, we isolate the foreground object using zero‑shot segmentation and generate context‑shifted variants with a structure‑preserving diffusion model, preserving object identity while varying the surrounding environment. We then introduce Cross‑Variant Self‑Supervised Learning, where variants of the same object under different backgrounds form positive pairs in a contrastive objective. This encourages the encoder to align object‑centric representations while suppressing background‑specific cues. Then, we fine‑tune the pretrained encoder using an ERM warm‑up followed by GroupDRO with layer‑wise learning rates. Experiments on distribution‑shift benchmarks demonstrate best worst‑group performance, achieving 92.5% on Waterbirds, 81.7% on MetaShift, and 87.4% on NICO++. Code: https://github.com/surajyadav‑research/GRSSL
Authors:Sergey Volkov, Yang Li, Ye Luo
Abstract:
Agent systems accumulate conflicting observations across branches, retries, and replicas, yet many practical memory layers still collapse disagreement behind overwrite rules that are difficult to inspect or correct. We present StateFuse, a conflict‑aware replicated memory contract built on standard OpSet/CRDT merge. StateFuse does not introduce a new join algebra; it defines an agent‑facing semantics layer with immutable history, explicit conflict objects, exact and semantic correction handles (claim_id / claim_ref), deterministic predicate contracts, and projection‑time resolution that cannot rewrite replicated state. We evaluate StateFuse against flat multi‑value, raw‑log, provenance‑style, and collapsed baselines under matched resolver and verification policies. On a 282‑question official conflict‑bearing MemoryAgentBench slice, the compared methods tie on answer accuracy, but conflict‑preserving surfaces keep contradictions visible while collapsed surfaces do not. In a controlled agent loop with uniform verification, preserving ambiguity enables safer abstention and correction than early collapse. A correction‑handle ablation further shows that semantic handles matter when exact prior identifiers are unavailable. The resulting claim is narrow: StateFuse is best supported as a safer public memory contract for contradiction surfacing, abstention, and auditable correction, not as a universal accuracy gain.
Authors:Yunkyu Lee, Woohyeok Kim, Sunghyun Cho
Abstract:
Defocus blur degrades fine image structures and limits visual perception, which can adversely affect downstream vision tasks. Although recent deep learning deblurring methods have achieved strong performance, their effectiveness depends on training data and often degrades across cameras and lenses due to limited optical diversity and realism in existing datasets. In this paper, we propose a pipeline for synthesizing realistic defocus deblurring datasets for diverse compound lenses. It integrates efficient wave‑optics PSF computation via Debye CZT propagation, depth‑aware defocus rendering with occlusion handling, and blur synthesis in the radiometrically linear space with camera ISP simulation. This unified pipeline enables the scalable generation of photorealistic defocus datasets with diverse lens characteristics. Using our pipeline, we generate CLDefocus, a large‑scale synthetic dataset containing lens‑diverse defocus image pairs. We further analyze the limitations of real‑captured defocus datasets and show that such imperfections can bias full‑reference evaluation. Extensive experiments demonstrate that models trained on CLDefocus achieve improved cross‑device generalization compared to models trained on existing real and synthetic datasets.
Authors:Gunner Levi Howe
Abstract:
The Minkowski functionals of a field's excursion sets ‑‑ area, boundary measure, and Euler characteristic ‑‑ describe its level‑set morphology; the Euler characteristic is the cheapest handle on topology. We derive smooth Monte‑Carlo estimators for all three of a continuous neural field, evaluated at scattered points via the co‑area formula and Gauss‑Bonnet, using only autodiff: no grid, no complex, no persistence. The estimator is accurate to 1‑3% against exact topology in 2D and 3D, and costs about 3 ms per iteration where a persistent‑homology (PH) loss on a cubical grid costs 650‑1000 ms ‑‑ a 250x gap. We establish four design rules without which these losses silently fail: a dense level ladder (invariants are flat in the parameters away from transitions), a C^2 backbone (ReLU nets hide curvature in kinks), the full Minkowski vector (Euler characteristic alone is an alternating sum, gamed by debris‑hole cancellation; pricing perimeter closes the channel), and sampling‑scale coverage. In 2D the vector‑valued cap is the only method in a controlled comparison that both repairs topology (3/3 seeds) and preserves fidelity ‑‑ uniform smoothing repairs at 11‑17x the fidelity cost, and the Euler term alone repairs nothing. In 3D neural‑SDF fitting, however, a failure mode we believe general to any sampled soft topology objective appears: gradient descent adversarially hides topological noise below the sampling density, where the estimator is blind ‑‑ spurious‑feature counts are invariant to 4x more samples, and closing the window needs cubically many points, erasing the cost advantage. A grid‑based PH baseline, whose complex is the evaluation resolution, solves the same benchmark (4/9 exact; median b_1 error 1 vs. ours above 10^4). The 250x cost of persistence is, at present, the price of having no null space. We release estimators, receipts, and benchmarks.
Authors:Sumit Chongder
Abstract:
Real‑time decoding is a major bottleneck in scaling quantum error correction (QEC) from noisy intermediate‑scale quantum (NISQ) devices to fault‑tolerant quantum computing. We present an adaptive confidence‑gated decoding framework for the rotated surface code that treats decoding as a two‑stage inference problem. A lightweight feed‑forward neural network performs fast‑path decoding for the majority of syndrome measurements, while only low‑confidence predictions are escalated to a minimum‑weight perfect matching (MWPM) refinement stage. We benchmark the framework on rotated surface codes with distances d \in \3,5,7,9,11\ under circuit‑level depolarising noise using the Stim stabiliser simulator. The evaluation characterises logical accuracy, confidence‑controlled accuracy‑latency trade‑offs, decoding throughput, per‑shot latency, and decoding‑graph resource scaling. Routing only 3.3%‑6.2% of syndromes to the refinement stage improves logical accuracy from 99.21% for the neural‑only baseline to 99.81% at a confidence threshold of 0.95 while incurring only a bounded increase in average decoding cost. Neural‑decoder throughput saturates near 4.6 × 10^5 samples s^‑1 at batch size 512 on commodity CPU hardware, indicating that the neural fast path is not the dominant throughput bottleneck beyond code distance d=7. We release the complete benchmarking pipeline, trained models, raw benchmark data, and source code, and explicitly distinguish the experimentally validated contributions from the broader hardware‑aware QEC co‑design roadmap, including hardware‑constrained code discovery, GPU‑accelerated inference, and multi‑noise optimisation, which remain directions for future work.
Authors:Praneeth Narisetty, Uday Kumar Reddy Kattamanchi, Shiva Nagendra Babu Kore
Abstract:
Dilution refrigerators are the enabling infrastructure of superconducting quantum computers, yet their fault diagnosis is still dominated by threshold alarms that report that something is wrong, not what. We present Onnes, a physics‑grounded digital‑twin simulator of a dilution refrigerator (a forward physics model with a learned real‑fridge noise fingerprint) that drives a live multi‑agent LLM operations layer, and use it for a controlled head‑to‑head between a zero‑shot LLM agent panel and a supervised ML classifier on cryogenic fault diagnosis. The twin couples a real dilution‑cooling floor, a noise‑and‑correlation fingerprint learned from real BlueFors logs, and six physics‑grounded fault classes, three engineered to overlap on temperature but separate on flow and pressure. Across a 1000‑turn evaluation the zero‑shot panel shows no significant difference from the classifier on detection but trails on classification, its errors concentrating on the confusable faults. Curated contrastive few‑shot demonstrations and self‑consistency voting then raise classification accuracy from 0.685 to 0.990, matching the supervised classifier (0.985) with no parameter updates and six labeled demonstrations; an ablation attributes the gain almost entirely to the demonstrations. Run as a continuous monitor across a nine‑run fault‑by‑seed sweep, the agent catches every developing fault within one poll interval, and a confidence gate suppresses pre‑onset false alarms whose rate is backend‑dependent. As a first sim‑to‑real check, a detector trained purely on real BlueFors telemetry posts a real‑hardware false‑alarm rate of 6.4% and 100% recall on physics faults injected onto real held‑out windows. All numbers are drawn verbatim from released run logs.
Authors:Peize Liu, Zhe Tong, Chen Feng, Shaojie Shen
Abstract:
Reliable calibration of multi‑fisheye camera systems remains challenging as rig size, camera arrangement diversity, and field of view increase. Existing pipelines can jointly optimize intrinsics, extrinsics, and target poses, but their success still depends heavily on empirical capture rules and the quality of the observations supplied to the solver. This paper studies this dependency through a failure‑oriented analysis. We reveal that calibration failures are not sufficiently explained by detector recall loss or global image‑plane distribution imbalance. Instead, the dominant failure factor lies in intrinsic initialization: observations with limited radial span couple focal scale with fisheye projection‑shape parameters, producing ill‑conditioned updates. Guided by this insight, we propose CO‑Calib, a plug‑in calibration‑data construction framework that combines a robust learning‑based target detector with an error‑analysis‑guided frame selector. CO‑Calib constructs initialization‑friendly anchors, co‑visible multi‑camera constraints, and coverage‑completion frames without changing the existing calibration workflow or optimization backend. Extensive experiments on synthetic and real multi‑fisheye systems demonstrate that CO‑Calib improves the overall success rate from 68.1% to 99.3%, increases extrinsic accuracy, and augments real‑world calibration stability. The source code will be made publicly available at https://github.com/HKUST‑Aerial‑Robotics/CO‑Calib.
Authors:Yimeng Zhang, Yingying Zhuang, Ziyi Wang, Yuxuan Lu, Pei Chen, Aman Gupta, Zhe Su, Ming Tan, Zhilin Zhang, Qun Liu, Manikandarajan Ramanathan, Rajashekar Maragoud, Edward Vul, Jing Huang, Dakuo Wang
Abstract:
Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self‑refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token‑level scores lack semantic coherence, while sequence‑level scores fail to localize errors. We formalize Span‑Level Uncertainty Estimation (SLUE), a new task that targets the natural granularity for uncertainty: semantically coherent text spans, each conveying a single assessable unit of meaning. To address this task, we introduce SPANUQ, a lightweight probe that distills the uncertainty knowledge from expensive multi‑sample inference into a single forward pass over LLM hidden states. SPANUQ employs a DETR‑style span decoder to simultaneously detect spans and estimate their uncertainty via a Mixture of Beta distribution, trained with a principled combination of Beta NLL regression and contrastive ranking objectives. We construct SPANUQ‑BENCH, the first span‑level uncertainty benchmark comprising 20K prompts, 293K annotated spans, and continuous soft labels derived from multi‑sample claim verification. Experiments on five LLM backbones show that SPANUQ consistently achieves the best span‑level uncertainty quality, outperforming the strongest probe baseline and all sampling‑based methods while being 10‑20x faster. Its DETR‑based span detector attains 0.910 F1, surpassing the best heuristic by 39.4%, enabling precise error localization that sequence‑level methods cannot provide. The framework generalizes across five LLMs spanning two model families.
Authors:Zhiwei Yang, Yuanchen Wu, Nan Zhang, Yucong Meng, Ke Yan, Shouhong Ding
Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated strong perception and reasoning capabilities. However, most existing models focus on isolated objects and neglect structured relationships for efficient target navigation, limiting their performance on visually intensive tasks. To address this challenge, we introduce Scene Graph Thinking (SaGe), a novel paradigm that enables fine‑grained and structured visual reasoning through explicit scene‑graph representations. Specifically, we first introduce an automated data engine that converts flat image‑text corpora into structured scene graphs, where hierarchical entities constitute the nodes and diverse visual relations define the edges. Building upon this, we construct 120K high‑quality training data by sampling reasoning traces from scene graphs. Then, two‑stage graph‑aligned post‑training paradigms are introduced, where supervised fine‑tuning internalizes MLLMs with structured reasoning, and subsequent reinforcement fine‑tuning proposes node‑as‑proxy graph rewards to consolidate efficient graph exploration. With curated data and graph‑aligned training, our approach achieves significant improvements across eight multimodal benchmarks, demonstrating strong effectiveness on fine‑grained perception and reasoning tasks. Code is available at https://github.com/zwyang6/SaGe.
Authors:Yufeng Wang
Abstract:
LLM systems for scientific discovery increasingly assist with ideation, literature synthesis, experiment planning, and report generation, but the first research question they propose can remain difficult to audit: it may sound plausible without exposing the mechanism, falsifier, or assumption that a scientist should inspect. We introduce FirstResearch, a first‑principles research‑question formation framework for scientific LLM agents whose core artifact is a structured Research Question Certificate. The certificate records primitive definitions, assumptions, a mechanism model, a tension or contradiction, a falsifiable hypothesis, a minimal decisive test, and a failure update rule, making the proposed question inspectable before downstream execution. On ten LLM‑agent research topics, FirstResearch outperforms controlled prompt‑level baselines inspired by AI co‑scientist, Agent Laboratory, and AI Scientist‑v2 under a primary DeepSeek‑blind‑judge protocol. A Gemini‑2.5‑Flash independent‑judge rescore of the same 40 baseline packages preserves the system‑level ranking, with FirstResearch scoring 4.86/5 versus 4.38/5 for the strongest baseline and Pearson agreement of 0.865 on average score. A one‑repeat ablation checkpoint further suggests that the certificate‑centered core is the strongest component: certificate‑only scoring reaches 4.90/5 under DeepSeek and 4.88/5 under Gemini, while removing certificates drops below 1/5 under both judges. These results are preliminary and use LLM judges rather than human domain experts, but they support a narrow scientific‑discovery claim: explicit derivation constraints are a promising mechanism for making LLM‑generated scientific questions more auditable. Code, prompts, saved outputs, and reproduction scripts are available at https://github.com/louiswang524/FirstResearch.
Authors:Runze Cheng, Yicheng Zhan, Josef Spjut, Kaan Akşit
Abstract:
Gaussian‑based image representations effectively model image content using compact parametric primitives while preserving high visual fidelity, yet storing a large number of floating‑point parameters per primitive degrades rate‑distortion efficiency at higher fidelity targets. To improve the rate‑distortion performance in Gaussian representation, we present our Cluster‑Guided Vector Quantization (CGVQ), a Gaussian primitive based image compression method. Our key idea is to partition Gaussian parameters further into homogeneous groups prior to quantization, enabling higher compression efficiency and accurate parameter reconstruction. In practice, our extensive experiments show that CGVQ decreases the bpp by 20% with respect to our baseline, while maintaining on‑par visual quality
Authors:Illia Dovhoshliubnyi, Nima Soroush, Ashkan Sami, Alexander Brownlee
Abstract:
AI coding agents are black boxes: we cannot inspect how they generate code, but we can inspect what they change. This distinction matters for search‑based software engineering (SBSE), where techniques such as genetic improvement (in the performance‑optimisation application we study) depend on mutation operators that reflect how code is actually transformed. Fewer than 1% of the 33,596 agent PRs in AIDev‑pop target performance, making each case a rare window into otherwise opaque agent behaviour. We classify 1,254 performance‑relevant diff hunks from 216 of these PRs, spanning five agent systems, against the 18‑category syntactic mutation taxonomy of Even‑Mendoza et al. (2025) using a dual‑LLM intersection pipeline. Three categories dominate: name modification (37.0%), object creation (26.4%), and type change (22.7%), a profile markedly different from prior GI corpora where no change accounted for 84%. Each agent's deployed system commits to a distinctive mutation vocabulary, and each performance strategy activates a largely disjoint category subset. Agent identity and target strategy are therefore informative priors that narrow the effective SBSE operator space. Replication package: https://github.com/5uper6rain/ssbse‑challenge‑2026
Authors:Clemens Walter Koprolin, Leonardo Trentini, Benedikt Soja, Mennatallah El-Assady, Christina Humer
Abstract:
Weather and climate foundation models produce high‑dimensional forecasts whose learned relationships are difficult to inspect with static plots alone. GeoXplain is an interactive Python‑based visualization toolkit for exploring geospatial attribution maps across climate variables, atmospheric pressure levels, and forecast time. The toolkit accepts attribution bundles containing attribution grids together with corresponding metadata and renders them in a notebook widget or browser with map and globe modes, linked timelines, pressure‑level controls, target annotations, and optional physical‑field overlays. We frame GeoXplain as a model‑agnostic earth‑system visualization toolkit and present the GeoXplain Aurora Adapter as its first computation backend. The adapter computes explanations for the Aurora foundation model, either in a local GPU process, through a GPU listener, or through a SLURM‑backed listener, while preserving the same Python call site for analysts. It currently supports gradient saliency, Integrated Gradients, RISE, ViT‑CX, multi‑frame saliency and Integrated Gradients rollouts, and retrieval of ERA5 overlays. GeoXplain can be installed as a PyPI package with pip install geoxplain. The code is open‑source and available at https://github.com/clemenskoprolin/geoxplain.
Authors:Hanan Gani, Guy Pulik, Daniel Rosenfeld, Duncan Watson-Parris, Salman Khan
Abstract:
High‑resolution satellite imagery is critical for observing fine‑scale cloud structures that inform weather modification strategies like cloud seeding for rain‑enhancement. However, the spatial resolution of current geostationary and polar‑orbiting satellites is often insufficient for capturing small cloud features. Current super‑resolution methodologies are suited for natural images and, therefore, struggle to generalize to satellite‑captured spectral images of cloud cover. To address this, we propose a two‑stage diffusion‑based super‑resolution framework to enhance the resolution of multi‑spectral cloud microstructures by a factor of 4×. Specifically, we use inverse diffusion to recover the high resolution properties from low resolution. Stage 1 utilizes real‑world paired data to learn robust degradation handling and inter‑sensor alignment, while Stage 2 employs a self‑supervised internal downgrading of high resolution data to refine structural learning and texture synthesis. Our approach outperforms the state‑of‑the‑art transformer and diffusion‑based baselines in both reconstruction accuracy and visual quality. We demonstrate that the two‑stage method better captures fine cloud microstructures (e.g. convective turrets and cloud gaps) that are crucial for effective cloud seeding decisions. Ablation studies confirm the complementary benefits of the two stages: Stage 1 excels in coarse structural fidelity, while Stage 2 contributes enhanced detail and realism. These results highlight a practical path toward improving cloud microphysics analysis and as a step towards utilizing AI for climate and sustainability. Our code and models are publicly available at: https://github.com/hananshafi/superresolution‑cloud‑microphysics.
Authors:Abhash Shrestha, Subigya Gautam, Anu Sapkota, Sanju Tiwari, Tek Raj Chhetri
Abstract:
Artificial intelligence increasingly mediates consequential decisions in healthcare, law, and public services, and the field has responded with an extensive methodology for measuring and mitigating bias. Yet the fairness definitions, benchmarks, and debiasing frameworks on which this methodology rests are treated as universal while being produced by a research community whose composition has never been characterized. We show that the AI bias research are structurally concentrated, and that this concentration is greatest, geographically, in precisely the domain the rest of the field inherits from. Analyzing 692 publications spanning five thematic domains, combining bibliometric analysis with semantic clustering, we find that research activity is dominated by a small set of countries, institutions, and authors, with the United States leading publication output and collaboration networks across every domain and most strongly in general fairness and bias mitigation, the largest, most‑cited domain with meaningful representation across all four semantic clusters. Low‑ and middle‑income countries remain largely absent from the community and its collaboration networks, and citation influence is highly skewed (median = 9; mean =93.5 ), indicating that a small fraction of publications disproportionately shapes the field. Because the general‑fairness domain supplies the definitions and benchmarks that application areas apply, concentration of research effort in this foundational domain propagates across AI bias research as a whole ‑ raising the concern that mitigation methods developed and validated within a narrow set of contexts may not generalize to all populations and settings where AI is deployed. We provide an interactive atlas for continuous monitoring of the field's structure.
Authors:Chakshu Gupta
Abstract:
The flip graph of an origami crease pattern has the flat‑foldable mountain‑valley assignments as vertices, and an edge joins two of them that differ by a single face flip. A basic invariant of this graph is the degree sequence, which counts the vertices of each degree. On the m× n Miura‑ori, this sequence is known as a bivariate polynomial only for small degrees, each count obtained by a separate argument. This paper gives one uniform construction that expresses, for every degree d, the number of degree‑d vertices as a single symmetric polynomial in (m,n) for all sufficiently large m,n. Subject to a single degree bound, this polynomial has total degree d‑2, growing for d\ge5 as an explicit multiple of m^d‑2+n^d‑2; the bound is proved here when the count splits into independent row and column factors, and open otherwise. The region is m,n\ge\max(d‑1,2); the polynomials are computed in closed form through d=10, and the bound is verified in every case through d=7. Below this region, the count departs from the polynomial by a correction whose leading coefficient, through degree eleven, is ‑4 times a Baxter number. Each such polynomial thus counts the Miura‑ori's flat‑foldable assignments admitting exactly d single face flips.
Authors:Sai Varun Kodathala
Abstract:
AI agents issue tool calls on the basis of text they cannot verify, so any party who controls part of the context can forge the appearance of authority. I evaluate 15 contemporary language models against eight attack scenarios derived from a published corpus of real agent incidents and find that refusal varies from 100% down to 38% across fully evaluated models; the most expensive model refused only half of the attacks despite a twentyfold price spread. I present aiAuthZ, an authorization gateway that moves the safety decision off the agent's host. Before a tool call executes, the gateway verifies caller identity with a per‑message HMAC‑SHA256 signature bound to a single‑use nonce and a timestamp window, and it evaluates a role‑based and argument‑level policy that the agent can neither read nor modify. Every decision joins a SHA‑256 hash‑chained audit log, and each accepted message yields an HMAC‑authenticated QR receipt that achieves 94% mean verification across eight transmission channels, with zero forgeries accepted in 25 wrong‑key trials. With the gateway in place, residual attack success falls to 0% for all 15 models at no more than 0.03 ms of added decision latency. On the AgentDojo banking suite, aiAuthZ blocks all seven attacker‑directed tool calls the evaluated agents emit, at the cost of one legitimate first‑time payment, while a spotlighting baseline allows two injections to succeed. Across nine in‑scope case studies from the same incident corpus, aiAuthZ blocks nine of nine, against four of nine for a policy baseline without identity binding. The gateway does not prevent a model from being deceived; it prevents a deceived model from acting beyond the verified user's authority on every call routed through it. The implementation and all experiments are released at https://github.com/Sports‑Vision‑Inc/aiAuthZ.
Authors:Chang Nie, Jiaju Wei, Junlan Feng, Chaoyou Fu, Caifeng Shan
Abstract:
Agentic video understanding equips models with long‑term memory to autonomously process and respond to continuous, long‑horizon multimodal streams. However, advanced video agents often rely on ``detective‑style'' iterative reasoning for action control (e.g., \mathttsearch) and evidence aggregation, incurring prohibitive costs and latency. We argue that such heavy reasoning primarily compensates for the lack of global context and semantic misalignment in retrieval. This paper introduces Light‑Omni, a multimodal agent framework for reflexive and lightweight video understanding. It achieves this through dual contextual states that instantly build the required context in a single forward pass. First, we maintain a global state, a finite‑sized multimodal script continuously consolidated from episodic memory, serving as the global context for Light‑Omni. Through hierarchical merging, it preserves recent details while summarizing past events. Second, conditioned on this global context, we generate a parametric latent state that directly drives autonomous actions and produces retrieval embeddings, with minimal latency. Benefiting from this coupled design, Light‑Omni achieves semantically aligned retrieval and reflexive responses while avoiding iterative reasoning. Extensive experiments validate the effectiveness of Light‑Omni across multiple video benchmarks. Notably, it outperforms M3‑Agent with an average 2.4% accuracy gain, a 12.1× speedup, and a 2.6× improvement in GPU memory efficiency. Furthermore, it serves as a memory system to enhance both the performance and efficiency of existing MLLMs. Project page: https://clare‑nie.github.io/Light‑Omni.
Authors:Kazumi Kasaura, Kei Tsukamoto, Kento Mori, Risa Mizuno, Takahiro Namatame, Yuta Oriike, Masaya Taniguchi, Sho Sonoda, Hayata Yamasaki
Abstract:
Quantum information theory is built on entropic quantities; among them, the sandwiched Rényi relative entropy is a fundamental divergence with various applications, and its data processing inequality (DPI) under quantum channels is a cornerstone result. In this work, we present a Lean 4 library for quantum information, designed as a reusable formal infrastructure for theoretical analysis. As a central demonstration of the library, we formalize the DPI for the sandwiched Rényi relative entropy for positive semidefinite operators on finite‑dimensional quantum systems. The library provides a basis‑independent operator‑theoretic framework for finite‑dimensional quantum mechanics compatible with the standard mathematical library Mathlib, including reusable interfaces for finite‑dimensional systems, states, channels, tensor products, partial traces, Choi operators, Kraus representations, and Stinespring representations. It also builds infrastructure for noncommutative trace inequalities, including operator monotonicity and convexity via the real continuous functional calculus, block‑operator positivity, Hilbert‑Schmidt operator spaces, Jensen's operator inequality, generalized perspectives, operator power means, and Lieb‑Ando trace inequalities. On top of this framework, we formalize entropy‑specific ingredients for the DPI: variational formulas for the sandwiched quasi‑entropy via Young and reverse‑Young inequalities, tensor‑product compatibility of real powers, and Haar measures on unitary groups. Together, these components yield a Lean formalization of the DPI, give strong subadditivity as a corollary, and provide the last missing component needed to complete the Lean formalization of the generalized quantum Stein's lemma. More broadly, the development provides machine‑checkable foundations for future formalized and AI‑assisted research in quantum information theory.
Authors:Ahmed Y. Radwan, Fahad Syed Muhammad, Matthew Baker, Hina Tabassum
Abstract:
Accurate and timely channel state information (CSI) is essential for next‑generation wireless systems, yet existing works treat CSI compression and CSI prediction as separate problems, both in academia and in current 3GPP studies. Consequently, channel aging remains insufficiently addressed within standardized CSI feedback pipelines. In this article, we propose a unified compression‑prediction framework that integrates Contrastive Predictive Coding (CPC) directly into the 3GPP‑compliant CSI compression architecture. Instead of predicting high‑dimensional CSI matrices, our approach forecasts future latent representations and jointly optimizes reconstruction fidelity and temporal predictive coherence via a combined 1‑SGCS and InfoNCE objective. This design enables temporal representation learning without increasing feedback overhead. We present two variants: CPC‑before‑Compression, which performs autoregressive modeling on encoded features prior to quantization, and CPC‑after‑Compression, which shifts temporal modeling to the base‑station to reduce the complexity of users' devices. Evaluations on 3GPP‑compliant datasets from Nokia, Oppo, and CATT show that CPC‑before‑Compression achieves over 90% reconstruction accuracy with 32x lower decoder GFLOPs than the 3GPP baseline, while CPC‑after‑Compression preserves an identical encoder footprint and the same 64‑bit feedback overhead. By unifying compression and prediction within a standardized pipeline, the proposed framework provides an age‑aware, computationally efficient CSI feedback solution. The source code is publicly available at: https://github.com/AhmedRadwan02/cpc‑3gpp
Authors:Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, Zhilong Zhang, Zheng Jiang, Bingxiang He, Wei-Ying Ma, Ya-Qin Zhang, Hao Zhou
Abstract:
Reinforcement learning with verifiable rewards (RLVR) is a powerful recipe for improving language‑model reasoning, but it is expensive to repeat on every new strong model because the target model must generate many rollouts during training. As models scale, post‑training itself becomes a bottleneck. We study a weak‑to‑strong alternative: run RL on a smaller model where rollouts are cheaper, then reuse what that RL run learned to improve a stronger target model. Directly distilling the post‑RL weak teacher is not enough, because the teacher's final policy mixes useful RL gains with the limitations of the smaller model. We propose Direct On‑Policy Distillation (Direct‑OPD), which transfers the teacher's RL‑induced policy shift instead. Direct‑OPD compares the post‑RL teacher with its own pre‑RL reference and treats their log‑ratio as a dense implicit reward for the student. In plain terms, the checkpoint pair tells us which actions RL made the weak model more or less likely to take, and Direct‑OPD applies that signal on the stronger student's own on‑policy states. This directly reuses the weak model's RL supervision signal without running sparse‑reward RL on the target model. Empirically, Direct‑OPD consistently leverages weaker teachers to improve stronger target models; notably, it boosts Qwen3‑1.7B from 48.3% to 58.3% on AIME 2024 in just 4 hours on 8 A100 GPUs. It outperforms step‑matched direct RL and enables the sequential composition of multiple policy shifts. Our results show that RL outcomes can be reused across model scales as implicit reward signals, not merely as final models to imitate.
Authors:Jacky Kwok, Shulu Li, Pranav Atreya, Yuejiang Liu, Yixing Jiang, Chelsea Finn, Marco Pavone, Ion Stoica, Azalia Mirhoseini
Abstract:
Scaling pre‑training, post‑training, and test‑time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM‑as‑a‑Verifier, a general‑purpose verification framework that provides fine‑grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM‑as‑a‑Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost‑efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM‑as‑a‑Verifier achieves state‑of‑the‑art performance on Terminal‑Bench V2 (86.5%), SWE‑Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine‑grained signals from LLM‑as‑a‑Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM‑as‑a‑Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.
Authors:Xinze Li, Yiyuan Wang, Pengxu Chen, Weifeng Su, Weisi Lin, Wentao Cheng
Abstract:
Streaming 3D reconstruction relies on a compact recurrent scene state to process long image streams in linear time and bounded memory. However, repeated updates can gradually corrupt this state, causing reliable historical information to be overwritten by noisy or ambiguous observations. We introduce ReCal3R, a reliability‑calibrated learning rate method for recurrent 3D reconstruction. Instead of directly applying a candidate learning rate, our method estimates state token reliability from the maintained scene state and uses it to calibrate a candidate learning rate derived from token alignment, state reconstruction residual, and recent update pressure. The resulting token‑wise learning rate interpolates between a conservative base rate and the candidate rate, suppressing aggressive updates on unreliable tokens while preserving adaptation to informative frames. Applied to CUT3R as a training‑free calibration rule, ReCal3R reaches strong performance on long sequences in pose, depth, and reconstruction quality, including a 3.7× reduction in ATE, with comparable runtime and memory. Code is available at: https://github.com/Powertony102/ReCal3R.
Authors:Andrew Snowdy, Ananya Trivedi, Sarvesh Prajapati, Lorena Maria Genua, Taskin Padir
Abstract:
In this work, we focus on the scenario of a robot‑assisted emergency evacuation. We consider two capabilities relevant to such a setting. The first is opening doors ahead of the people being evacuated, so that their path toward an exit stays clear. The second is retrieving rescue equipment and delivering it to the emergency responders carrying out the evacuation. From a systems perspective, this involves several tasks at once. The robot must locate ADA‑compliant door buttons and the rescue equipment it needs to retrieve. Additionally, it must remain aware of the people around it and adapt its behavior to them, so that it supports the evacuation rather than getting in the way. We address these demands with a behavior tree at the core of our framework. This structure is chosen for its ability to select high‑level tasks based on environmental triggers, and to extend to new situations as they arise. We evaluate the system in 105 trials on the Toyota Human Support Robot, across five hardware and three simulation scenarios. These trials capture the decisions the robot must make in this setting: whether to press a door button, yield to a nearby person, walk through a door someone else is holding, or first retrieve rescue equipment before traversing the door. Overall, the system completes 97 of the 105 trials successfully. These results suggest our framework provides a practical basis for robotic assistance in broader emergency response tasks. Code and video demonstrations are available at https://github.com/AndrewSnowdy/hsr_mm_control.
Authors:Yueyang Wang, Baolong Bi, Shuo Lu, Jingyuan Zhang, Jiajun Shi
Abstract:
Supervised fine‑tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target‑domain behavior at the cost of degrading pre‑existing capabilities. Standard cross‑entropy fine‑tuning promotes only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible alternatives, potentially distorting the rich local preference structure learned during pretraining. We first analyze next‑token predictions using Shannon and Renyi entropies, revealing that pretrained models exhibit a regular multimodal entropy structure. These entropy peaks correspond to varying numbers of plausible alternatives, indicating that the base model intrinsically encodes rich distributional knowledge beyond the single supervised token. Motivated by this observation, we propose LP‑SFT, a Local‑Preserving Supervised Fine‑Tuning objective designed to explicitly protect this inherent entropy structure. At each step, LP‑SFT constructs a local top‑K support of alternative tokens from the frozen base distribution. Crucially, it removes the supervised target token from this set to avoid conflicting with the cross‑entropy objective, and applies a locally normalized KL divergence to maintain the base model's relative preference structure among the remaining non‑label alternatives. Across mixed‑domain and single‑domain fine‑tuning experiments, LP‑SFT improves overall performance over vanilla SFT and recent SFT‑enhancement baselines, achieving the best balance between pass@1 accuracy and pass@k performance. These results suggest that local preservation helps mitigate capability degradation without collapsing sampling‑accessible diversity.
Authors:Guangting Zheng, Haojing Chen, Hao Li, Jingtao Zhang, Zhen Yang, Xiaosong Jia, Xue Yang, Shaofeng Zhang, Yanyong Zhang
Abstract:
While modern video diffusion models excel in visual fidelity, maintaining long‑range physical consistency remains a formidable challenge. Conventional pixel‑reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose VPT, a fine‑tuning framework for improving physical consistency in video diffusion models. VPT introduces a role‑aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality‑decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss‑weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross‑step auto‑guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4% in SA and 17.9% in PC on VideoPhy benchmark over Wan2.1‑T2V‑1.3B, and consistent improvements on VideoPhy‑2 benchmark. The project page is available at https://tom‑zgt.github.io/VPT.
Authors:Meng Du, Hongchang Chen, Ran Li, Junjie Zhang, Qi Ouyang, Shibo Zhang, Shuxin Liu
Abstract:
Rapid advances in AI video generation pose increasing security risks and call for reliable detectors with strong cross‑domain generalization. Although existing methods perform well under in‑domain evaluation, their performance degrades substantially on unseen generators. A key reason is shortcut learning, where detectors rely on domain‑specific bias rather than intrinsic forensic cues. To address this issue, we propose G2VD, a generalizable AI‑generated video detection framework based on counterfactual intervention and causal disentanglement. First, G2VD introduces a counterfactual intervention pipeline (CFIPipeline) that constructs counterfactual samples through VAE‑based reconstruction and subsequent frequency‑domain and pixel‑domain alignment, thereby weakening spurious correlations between domain‑specific bias and authenticity labels. Building on this intervention, we further design a causal disentanglement classifier that combines two domain‑anchored branches with complementary objectives and a constraint based on the Hilbert‑Schmidt Independence Criterion (HSIC), encouraging the causal and non‑causal representations to capture intrinsic forensic cues and domain‑specific bias, respectively. Experiments across four public datasets demonstrate strong cross‑domain performance and consistent gains over baseline methods. In the challenging GenVidBench setting, G2VD achieves over 90% overall ACC, with improvements of 0.194 in F1 and 0.104 in AUC over comparable state‑of‑the‑art methods, while using only 10% of the available training data. Code is available at https://github.com/DMOSCAR‑98/G2VD.
Authors:Sokipriala Jonah, Queen Moses, Abiola Babatunde, Michael Ajao-Olarinoye, Daniel Bammeke
Abstract:
Home Energy Management Systems (HEMS) can reduce residential electricity costs, but many require users to express everyday preferences as technical constraints. This paper presents a tool‑calling ReAct agent that converts natural‑language requests into schedules for multiple household appliances using half‑hourly Octopus Agile prices, weather forecasts, photovoltaic generation estimates, and household demand data. Five large language model backends are evaluated against a mixed‑integer linear programming benchmark across dynamic tariff conditions, constraint conflicts, weather‑aware scheduling, and a seven‑day rolling deployment. Native function calling achieves high scheduling success and near‑optimal cost on ordinary tariff days, whereas text‑parsed actions reduce reliability. Constraint‑conflict testing shows that low cost does not guarantee safe or feasible behaviour. Claude Sonnet 4.6 performs best in power‑cap and infeasibility scenarios, while Qwen‑3 achieves higher overall constraint compliance than GPT‑4o‑mini. The evaluation also identifies fabricated schedules, failed commitments, and reasoning‑to‑action failures in which models explain a deadline correctly but commit an invalid schedule. Weather‑aware scheduling reduces cost and increases solar self‑consumption under overcast conditions, but provides limited or adverse economic value under some dynamic‑price regimes. Across the evaluated seven‑day period, the agents capture 96.7‑98.0% of the savings available between an off‑peak timer and the MILP oracle and outperform the rule‑based policies. The results support LLM‑based HEMS orchestration, provided that every committed schedule is checked by an independent deterministic feasibility validator before actuation. Code and a live demonstration are available at https://github.com/sokistar24/ecohome‑experiments and https://www.ecohomeagent.com/.
Authors:Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, Songling Liu, Haowen Yan, Honghao Su, Zhiyang Dou, Kaixuan Wang, Dandan Zhang, Yunze Liu, Yan Qin, Qiwei Liang, Qiwei Wu, Zijian Lin, Wenwei Lin, Yuran Wang, Minghua He, Tianshu Wu, Ruihai Wu, Jingquan Zhou, Kai-Chong Lei, Haibao Yu, Yuanfeng Ji, Weiyang Jin, Guanyu Lin, Xiaofan Li, Qi Xiong, Renjing Xu, Zhongyu Li, Wenhao Chai, Enze Xie, Ziwei Wang, Yao Mu, Hao Dong, Wojciech Matusik, Mingyu Ding, Wenbo Ding, Ping Luo, Masayoshi Tomizuka
Abstract:
Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short‑horizon, or skill‑narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulation enables scalable feedback but misses physical deployment challenges, while real‑world evaluation is costly, time‑consuming, and difficult to reproduce. We introduce RoboDojo, a unified sim‑and‑real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real‑world tasks covering diverse and complementary manipulation capabilities. The simulation benchmark evaluates five dimensions: generalization, memory, precision, long‑horizon execution, and open‑vocabulary instruction following, while the real‑world benchmark exposes policies to challenging physical‑world deployment conditions. RoboDojo supports scalable evaluation through heterogeneous parallel simulation in Isaac Sim and provides RoboDojo‑RealEval, a reproducible real‑world evaluation system with remote cloud access, standardized hardware, scene reset, evaluation protocol, and deployment interface. Together with XPolicyLab, policies can be integrated once and evaluated across simulation and real‑world settings with minimal adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard and systematic analysis of current policy performance. The website is available at http://robodojo‑benchmark.com/.
Authors:Jiwon Kang, Heeji Yoon, Jaewoo Jung, Jaewon Min, Minkyeong Jeon, Biyeon Hwang, Sangwon Jung, Seungryong Kim
Abstract:
Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied. We investigate \boldsymbol\mathsftransferability in UMMs: whether training a capability on one task improves the same capability on the other without explicit supervision. Through controlled experiments, we empirically find that transferability depends on architecture‑models with fully shared transformer backbone and a unified visual encoder exhibit consistent cross‑task transfer, while loosely coupled designs show little or none. Leveraging this transferability, we propose a practical training strategy. The most straightforward way to improve a target generative capability (e.g., counting) is to fine‑tune generation directly, but this can degrade visual quality due to distribution shift. Instead, we train the corresponding understanding task and let it transfer into generation, which improves capability‑specific generative performance while minimizing distribution shift. We validate this across three capabilities‑counting, spatial relation, and text recognition/generation‑showing that cross‑task transferability can be systematically exploited in UMMs.
Authors:Zixiang Zhou, Zhentao Yu, Yifeng Ma, Hongmei Wang, Wenqing Yu, Cong Wang, Zilin Yang, Rui Chen, Jiarong Ou, Yezhou Liu, Yuan Zhou, Qinglin Lu
Abstract:
Subject‑driven and multi‑element video generation are central to controllable video synthesis, but existing methods still struggle to preserve identity consistency and model complex relationships among multiple subjects. In this paper, we propose Aura, a unified framework for high‑fidelity and identity‑consistent video generation. To better capture scene dynamics and subject interactions, we introduce AI director‑level captions that provide dense and structured descriptions of video content. We further leverage a vision‑language model (VLM) with learnable queries to extract multimodal semantic features from textual and visual references, covering both global semantics and fine‑grained visual cues. To bridge the representational gap between the VLM and the Diffusion Transformer (DiT), we design a two‑stage alignment strategy that progressively maps VLM features into the DiT feature space. For visual conditioning, we adopt token concatenation to inject reference information directly into the generation process. To distinguish heterogeneous subject types and reduce common copy‑paste artifacts, we develop a subject‑aware RoPE‑Shift mechanism. To further differentiate reference images of different categories, we introduce subject‑aware learnable tokens. In addition, we introduce Memory Tokens to balance the training signal across examples with different numbers of reference subjects. During inference, Progressive‑APG (Adaptive Prompt Guidance) further alleviates oversaturation and improves semantic alignment with user prompts. Finally, we build a high‑quality video‑subject image dataset through a dedicated data construction pipeline. Extensive experiments show that our method achieves state‑of‑the‑art performance on both single‑subject generation and more challenging multi‑element scenarios.
Authors:Jaeyeon Kim, Jewon Lee, Bo-Kyeong Kim
Abstract:
This report describes our approach to the Efficient Qwen Competition, where the goal is to enable low‑latency serving of Qwen3.5‑4B on a resource‑constrained NVIDIA A10G GPU. Our system combines a quantized target model with speculative decoding. To recover accuracy, we apply quantization‑aware distillation to the target model while retaining the original quantization grid. To speed up decoding, a block‑diffusion drafter specialized for the quantized target model is trained using a two‑stage procedure: first learning from the high‑precision target and then adapting to the low‑precision target. Because the drafter is invoked at every speculative decoding step, we further reduce its overhead with quantization and sliding‑window attention, preserving draft‑token acceptance while improving long‑context decoding latency. As a result, our submission achieves a 6.978× average speedup over the baseline while satisfying the required quality thresholds, ranking 3rd overall. We hope these results provide useful insights for practical LLM inference. The code and resources are available at https://github.com/nota‑github/adaptfm‑quant‑dflash
Authors:Aisha Alansari, Malak Alkhorasani, Hamzah Luqman
Abstract:
Recent hallucination detection techniques in large language models (LLMs) focus on directly extracting features from a model's internal representations and training a classifier on these features to detect hallucinations, demonstrating promising results. Notwithstanding this advancement, most internal‑state hallucination detection techniques have been explored predominantly in English, raising the question of whether such internal signals generalize across different languages and domains. To address this gap, we present CrossHallu, the first study to evaluate the cross‑lingual and cross‑domain generalization of hallucination detection using internal representations from six LLMs on the generative question‑answering task. We conduct a systematic Arabic <‑> English evaluation using TruthfulQA, an Arabic translated version of TruthfulQA, and HalluScore. This evaluation encompasses monolingual training and testing, cross‑lingual transfer, cross‑domain transfer, and combined cross‑lingual and cross‑domain transfer. The results reveal that internal‑state hallucination signals in LLMs transfer across languages and domains for most models, with cross‑lingual performance highly dependent on both class separability and language alignment in the feature space, whereas cross‑domain transfer within Arabic varies depending on the training and testing datasets used for the hallucination detector. The code is publicly available at https://github.com/aishaalansari57/CrossHal.
Authors:Zhengpeng Feng, Sadiq Jaffer, Ira Shokar, Jovana Knezevic, James Ball, Pedro Sousa, Mark Elvers, Madeline Lisaius, Clement Atzberger, Robin Young, Aneesh Naik, Niall Robinson, David Coomes, Anil Madhavapeddy, Srinivasan Keshav
Abstract:
Pixel‑wise Earth‑observation (EO) foundation models are now achieving state‑of‑the‑art performance via generated spatial embeddings. However, how these models scale and how best to spend a pretraining budget remain poorly understood. We present the largest controlled scaling study for EO to date: 395 training runs within a fixed pixel‑wise Barlow Twins family, each evaluated on 15 diverse downstream tasks. We find that pretraining loss barely predicts downstream performance (|Pearson r| < 0.2), so selecting models by loss wastes a large share of the compute. We also find that, as the training budget grows, the encoder and the data should grow together while the projector stays fixed, which gives a simple rule for allocating compute. Using this rule, we train a family of pixel‑wise teachers (0.5B, 1B, and 2B) and distil the largest into compact students for embeddings‑as‑data deployment. In aggregate, our 44‑million‑parameter distilled student outperforms every open and proprietary embedding product we test, several of them an order of magnitude larger. These students produce Matryoshka representations that are inexpensive to serve: a 16‑dimensional prefix keeps 92% of the full 128‑dimensional performance at 1/8 of the storage. Together, these results give a concrete, empirically grounded recipe for scaling pixel‑wise EO foundation models: train large encoders, select by downstream performance, and distil into flexible student models. We plan to release global 10 m annual embeddings covering 2017‑2025 as version 2 of the TESSERA foundation‑model embeddings product. All code is available at: https://github.com/ucam‑eo/tessera
Authors:Zhigang Yang, Huiguang Yao, Linmao Tian, Qiang Li, Qi Wang
Abstract:
Remote sensing imagery plays a crucial role in evaluating regional transportation capacity. However, existing segmentation datasets often lack diversity in object categories and scenes, limiting the ability of models to comprehensively evaluate trans portation capacity in real‑world scenes. To alleviate this gap, we construct a large‑scale and diverse dataset for transportation object segmentation, named as NWPU‑Traffic. This dataset encompass four traffic object categories (car, airplane, ship, and train) and a wide range of scenes from 49 cities across 7 countries, with instance‑level annotations to ensure precise segmentation of individual objects, which bridges critical shortcomings in resolution and scene diversity in existing datasets. Leveraging this dataset, we establish a benchmark with several popular segmentation networks. Furthermore, we propose a novel segmentation method that leverages spatial‑channel preserving feature interaction and an adaptive feature decoder, enabling robust segmentation across varying scales and complex environments. Extensive experiments and ablation studies validate the effectiveness of our approach. The dataset and code are publicly available at https://github.com/CVer‑Yang/NWPU‑Traffic.
Authors:Fatemah Alhamdoosh, Pietro Pala, Abduallah Mohamed, DK Arvind
Abstract:
Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement‑related conditions. Conventional clinical assessments are often intermittent and may fail to capture subtle temporal variations in motor behavior. While wearable IMUs and third‑person video have shown promise for objective motor assessment, third‑person recordings raise privacy concerns and require constrained acquisition setups. In contrast, egocentric vision provides a more naturalistic and privacyaware alternative. In this work, we introduce EgoInertia‑MI, a multimodal benchmark dataset combining synchronized egocentric video and wearable IMU signals for motor impairment analysis. The dataset contains 19 upper‑ and lower‑body activities performed by healthy volunteers simulating varying levels of motor impairment severity levels: no impairment, mild impairment, and severe impairment. We establish two benchmark tasks: action recognition and motor impairment severity estimation, and evaluate multiple unimodal and multimodal baselines. Experimental results show that egocentric video provides strong cues for motor impairment assessment, while multimodal fusion achieves the best overall performance, reaching 0.78 Macro‑F1 for severity estimation and 0.93 Macro‑F1 for action recognition. These findings highlight the potential of combining egocentric vision and wearable sensing for ecologically valid and privacy‑aware motor assessment. Code and data are available at:https://fatemah‑alh.github.io/EgoInertia‑MI‑Page/.
Authors:Mykhailo Poliakov, Nadiya Shvai
Abstract:
Multi‑Meta‑RAG improves retrieval for multi‑hop question answering by filtering a vector store on metadata (the news source) that it extracts from each query by prompting gpt‑3.5‑turbo. We show this proprietary, free‑form extractor can be replaced by a local, deterministic probe trained on the hidden states of a small open‑source language model. On all 2556 MultiHop‑RAG queries the probe reaches 90.9% set‑exact accuracy against 88.0% for a model‑free substring baseline and 80.9% for GPT‑3.5, a margin that comes entirely from null queries, on which GPT‑3.5 never abstains; on non‑null queries all three stay within about a point. Because the probe's output space is exactly the fixed 49‑source vocabulary, it cannot drift outside the allow‑list as the prompted model does. Three design choices make it work: selecting a shallow layer, mean pooling, and class‑imbalance‑aware multi‑label training over the long tail of sources. A 135M‑parameter model lands within ~1.5 points of a 1.5B one, so the filter is cheap to output: a partial forward pass through the first few layers plus one linear head, with no API. The code is available at https://github.com/mxpoliakov/Multi‑Meta‑RAG.
Authors:Siru Jiang, Jian Liang, Ran He, Tieniu Tan
Abstract:
Test‑time adaptation (TTA) has emerged as a popular paradigm for improving the performance of vision‑language models (e.g., CLIP) on downstream tasks. Among existing CLIP‑based TTA methods, Test‑Time Prompt Tuning (TPT) is a pioneering work that optimizes textual prompts using multiple test‑time augmentations and remains a strong baseline to date. In this work, we revisit TPT and reveal that its optimization can be interpreted as implicitly learning from self‑generated pseudo labels. Building on this perspective, we propose a unified self‑ensembling framework (USE) that ensures consistency between the optimization and inference stages. During optimization, we introduce a simple yet effective self‑ensembling (SE) strategy that emphasizes the test image itself over its augmented views adaptively to obtain more reliable pseudo labels. To fully exploit the potential of augmentations, we further apply the same strategy at inference time, unifying the objectives of both stages. Notably, SE can also act as a lightweight optimization‑free TTA method. Extensive experiments across multiple datasets demonstrate that SE and USE outperform their counterparts, respectively. Furthermore, SE yields consistent performance gains when integrated with existing TTA methods. The code is available at https://github.com/sirujiang/USE.
Authors:Jakub Zadrozny, Oisin Mac Aodha, Hakan Bilen
Abstract:
3D reconstruction of articulated objects from a single image is challenging because large training datasets with paired image and 3D supervision are difficult to obtain. Recent point map‑based methods achieve strong performance but rely on synthetic datasets rendered from manually created articulated 3D assets with carefully curated pose distributions. While camera viewpoints can be easily sampled, generating realistic object articulations remains costly and labor‑intensive. We propose a training framework that reduces this requirement by leveraging unannotated 2D images collections with only a single rigged canonical mesh per category. Starting from a weak 3D shape predictor trained on canonical‑pose renders, we iteratively estimate object articulation and camera pose by fitting the mesh to predicted point maps. The recovered articulations and viewpoints are then used to render updated synthetic training data, progressively improving the predictor. Despite using substantially weaker 3D supervision, our models achieve performance comparable with DualPM, which requires manually curated articulated training datasets.
Authors:Nitzan Hodos, Roy Amoyal, Lior Fritz, Ianir Ideses, Sagie Benaim, Netalee Efrat
Abstract:
Close‑up rendering, zooming into a scene well beyond any training camera, is important for virtual production and interactive 3D content, yet remains an open challenge. 3D Gaussian splatting (3DGS) enables high‑fidelity, real‑time novel view synthesis, but its rendering quality degrades at close range. Recent diffusion‑based methods that enhance the rendering by conditioning on reference images from the training set produce significant artifacts in this setting. We analyze this failure and identify its root cause: the scale gap between the close‑up and reference views. We show that the features in reference‑conditioned enhancement models are not scale‑invariant, causing cross‑view attention to retrieve incorrect correspondences when the same content appears at different scales, and that this mismatch cannot be corrected in latent space because the VAE encoder is not scale‑equivariant. Building on this analysis we introduce MACRO, Multi‑plane Attention for Closeup Render Optimization, a training‑free method for high‑quality close‑up novel view synthesis from 3DGS. MACRO resolves the scale gap by leveraging the scene's known 3D structure: it decomposes the close‑up into depth planes, crops and resizes references in image space to match the scale of each plane before encoding, and applies a depth‑aware attention mask so each token attends only to scale‑matched references. The method requires no architectural changes or additional training. We further contribute two new close‑up novel view synthesis benchmarks, the first standardized evaluation protocol for this setting, and demonstrate state‑of‑the‑art results on both, outperforming existing 3DGS and diffusion‑based methods on both reconstruction and perceptual metrics. Project page: https://nitzanhod.github.io/MACRO
Authors:Ido Amit, Ido Galil, Ran El-Yaniv
Abstract:
As LLMs generate increasingly long outputs, effective uncertainty estimation must identify errors at fine‑grained levels rather than discard entire responses. While such methods exist, evaluating uncertainty at any resolution (token to an entire generation) is challenging and highly sensitive to label imperfections, making zero‑noise benchmarks essential; yet, long‑form generation benchmarks tend to rely on fallible labels rather than deterministic ground truth. We introduce Single‑answer Atomic Long‑form Target (SALT), a benchmark of six procedurally generated tasks with single deterministic long textual ground truths, enabling unit‑level evaluation of correctness, calibration, and ranking without external judges. Equipped with SALT, our analysis of 50+ LLMs reveals key insights: We identify which confidence functions dominate each uncertainty aspect and show that confidence ranking largely breaks at atomic resolution, even when clearer separability emerges at coarser line‑level units. SALT further enables controlled atom‑level interventions throughout generation, revealing two separable drivers of future errors: propagation from corrupted prefixes, dominated by global context correctness, and bounded degradation from increasing answer‑context length. Finally, we demonstrate that reasoning, via Chain‑of‑Thought prompting or internalized through training, introduces a trade‑off, improving accuracy while degrading confidence ranking. These findings directly impact risk‑critical applications requiring reliable error identification and mitigation.
Authors:Yuhang Jiang, Guohui Deng, Miaozhong Xu, Chao Ruan, Jinling Zhao, Linsheng Huang
Abstract:
Referring remote sensing image segmentation isolates the object named by a natural‑language expression in an aerial image. Existing training‑free methods resolve the expression through implicit vision‑language activations or region‑text similarity, which gives weak control over the spatial, comparative, and ordinal relations that dominate aerial referring: they cannot represent constructions such as the largest ship or the second court from the left. We propose GeoSelect, a training‑free pipeline that reframes referring as the execution of a typed spatial program. A frozen, text‑only language model synthesises the expression into a small domain‑specific language, a well‑formedness checker accepts the program, and a deterministic executor runs it. The central abstraction is a single scored candidate set type under which every operator composes: continuous geometric fields realise position and proximity as dense pixel‑level maps, while discrete set and order operators add the extremum, ordinal, counted‑union, and relational constructions that fields alone cannot express. Because execution is explicit, every intermediate program, field, and ranking is inspectable, and a reliability ladder degrades any failing program to a field‑only special case, so every expression returns an answer. GeoSelect attains 58.86 mIoU on RRSIS‑D test and 55.27 mIoU on RISBench test, more than twice the best prior training‑free method on RRSIS‑D, with no referring supervision and on a single GPU. A controlled comparison with candidates and segmenter fixed attributes the gain to explicit execution, not the backbone; an oracle decomposition localises the residual gap to detection recall on RRSIS‑D and selection on RISBench, and an exposure audit confirms robustness to pretraining leakage. Code will be released upon acceptance at the project page https://avalon‑s.github.io/GeoSelect/.
Authors:Fuqiang Chen, Yifeng Wang, Hongpeng Wang, Yongbing Zhang
Abstract:
A unified multiplex virtual staining model enables scalable and non‑destructive multiplex analysis from H&E slides while promoting parameter efficiency, shared pathological knowledge, and consistent cross‑biomarker representations. However, in clinical practice, data for new biomarkers are typically acquired sequentially over time. Fine‑tuning on such temporally arriving data leads to severe performance degradation on previously learned biomarkers, as sequential optimization disrupts the structured relationships among biomarker representations in the latent space. To address this issue, we propose ContiStain, an IHC multi‑domain relational distillation framework for continual virtual staining. We first (i) construct a domain‑aware structured feature space using a mixture‑of‑experts (MoE) feature extractor to reduce representation interference across biomarker domains. Based on this stabilized feature space, we then (ii) propose a relation‑preserving distillation strategy that explicitly enforces the consistency of cross‑domain token‑level cosine similarity matrices between learned biomarker domains during continual adaptation. By maintaining cross‑domain structural coherence, ContiStain mitigates forgetting while retaining adaptability to new domains. Experiments on the MIST dataset under a four‑domain sequential virtual IHC staining setting show improved stability, reducing FID and ConchFID by 11.1 and 60.9 compared to sequential fine‑tuning, enabling scalable and robust multi‑domain virtual staining. Code is released at https://github.com/ccitachi/ContiStain.
Authors:Zhen Huang, Peicheng Xu, Junbiao Pang, Yulong Zheng
Abstract:
Sparse feature selection is critical for high‑dimensional machine learning, yet traditional \ell_1‑regularized methods are often brittle under observational noise and spurious correlations, leading to unstable feature supports and degraded generalization. Although adversarial training has been widely used to improve model robustness, its interaction with hierarchical sparse feature selection remains underexplored. In this work, we propose Adversarial LassoNet (AdLNet), a stability‑driven sparse feature selection framework that integrates input‑space adversarial perturbations with the hierarchical sparsity mechanism of LassoNet. We derive a tractable first‑order adversarial approximation under local smoothness assumptions and provide an NTK‑inspired spectral analysis to characterize how perturbation‑driven training can reduce gradient concentration. Experiments on high‑dimensional SERS data, six public benchmark datasets, and ColoredMNIST show that AdLNet maintains competitive sparse‑selection performance while improving out‑of‑distribution robustness by 4.4% and feature support reproducibility by 6.3% under nearly matched support sparsity on ColoredMNIST. On the high‑dimensional lung cancer screening dataset, AdLNet achieves a 5.3% test accuracy gain and a 6.0% AUC improvement over vanilla LassoNet. Code and dataset are available at https://github.com/719573/Adversarial‑LassoNet.
Authors:Senol Gulgonul
Abstract:
Converting a SPICE netlist into a human‑readable schematic is a longstanding problem in electronic design automation: simulators and machine‑learning pipelines readily produce netlists, but designers reason about circuits through diagrams. Recent learning‑based approaches translate netlists into schematics probabilistically, yet they provide no guarantee that the generated drawing preserves the original connectivity, and their accuracy degrades sharply as circuits grow. We present Weave, a deterministic converter that turns a SPICE netlist into an LTspice .asc schematic using a layered (Sugiyama‑style) graph layout, and that certifies every output by a round‑trip connectivity check: the generated schematic is re‑parsed into a netlist and compared, net for net, against the input. A result is reported as correct only when the two partitions are identical, giving a binary correctness certificate rather than a similarity score. Weave runs entirely client‑side as a single dependency‑free file and embeds a pin table for 5093 LTspice symbols. On the identical public Circuits‑LTSpice test set used by the state‑of‑the‑art LLM converter Schemato (117 circuits, netlisted with LTspice itself), Weave achieves 100% compilation and 100% round‑trip‑verified connectivity equivalence, compared with Schemato's reported 76% compilation and a graph‑edit‑distance similarity of 0.35; notably, 73% of that set exceeds the five‑component threshold beyond which Schemato reports losing connectivity accuracy. On a larger and harder corpus, the 3460 netlistable circuits of the official Analog Devices LTspice demo collection, Weave verifies exact connectivity for 88.4% of circuits, with the remaining failures concentrated in a single, well‑characterized class of dense multi‑pin power modules.
Authors:SungHun Kim, SeungJun Baek
Abstract:
Audio‑Visual Question Answering (AVQA) extends classical VQA by requiring joint reasoning over video and synchronized audio. However, many AVQA systems rely on deeply stacked layers of self‑ and cross attention across text, video, and audio. Such sequential stacking may incur loss of information such as subtle inter‑modal cues over the layers, causing errors to accumulate across sequential attention layers during the fusion. We introduce Q‑TriM which performs multi‑modal fusion in a shallow and parallel manner instead of a deep and sequential manner. For Q‑TriM, we propose a novel framework for attention operation incorporating video and audio conditioned on text. As a result, we obtain not only standard cross attention outputs but also Tri‑Modal Attention representations in which Query, Key, and Value come from distinct modalities. These attention representations are combined in parallel at a single stage, thus avoiding the multi‑modal fusion with deep stacks in order to mitigate error accumulation and depth‑induced issues. Q‑TriM achieves state‑of‑the‑art performance on three AVQA benchmarks, including substantial gains on MUSIC‑AVQA‑R, which demonstrates its robustness and out‑of‑distribution generalization. Code is available at https://github.com/Sunghun95/Q‑TriM
Authors:Zhenyu Sun, Xiaohan Zhang, Qi Liu, Huan Wang
Abstract:
Challenges remain in ego‑centric 3D scene generation due to limited view overlap and the dominant influence of individual perspectives on scene interpretation. These factors hinder the creation of viewpoint‑consistent and semantically aligned visual content, as well as the construction of accurate geometric structures. In this paper, we propose CGGS, a text‑to‑3D framework aiming to enhance 3D‑content‑awareness and address geometric distortions in ego‑centric scene generation. Firstly, the Ego‑centric Generator is proposed by fine‑tuning a Multi‑View Latent Diffusion Model with consistency‑augmented loss to generate consistent, high‑fidelity 2D content aligned with textual descriptions. Then, Layout Decorator leverages optical flow and point‑track correspondence to estimate depth, therefore producing dense point clouds as coarse layouts from the ego‑centric 2D priors. Building on this initialization, Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy‑based Mutual Information Depth Loss (MID) combined with a hierarchical optimization scheme for improving visual quality and geometric structure. Comprehensive experiments demonstrate that CGGS outperforms previous methods in generating coherent and accurate text‑driven 3D scenes. Project page: [https://cggs‑26.github.io/cggs26/](https://cggs‑26.github.io/cggs26/).
Authors:Octavian Gîngu, Stelian Spînu
Abstract:
Indoor search‑and‑rescue (SAR) operations often require rapid situational awareness where GNSS signals are unavailable and human access is difficult or hazardous. While most autonomous aerial systems rely on LiDAR, stereo vision, or specialized depth cameras, such solutions increase both hardware complexity and deployment costs. This paper presents a complete autonomous indoor navigation framework for low‑cost unmanned aerial vehicles based exclusively on monocular vision. Implemented on a DJI Tello platform, the system combines monocular depth estimation using Depth Anything V2 with classical computer vision and lightweight deep learning models for scene understanding, victim detection, and hazard recognition. The framework consists of two independent behaviors: (i) corridor exploration with automatic door detection, room entry, OCR‑based room identification, and victim inspection; and (ii) autonomous stair ascent based on TRISTAR (TRI‑Signal STair Ascent Recognition), a novel triple‑sensor fusion method that integrates structural cues (Sobel filtering), texture analysis (multi‑scale Gabor filtering), and geometric depth from monocular depth estimation. Evaluation used real indoor flights in a university building. Depth calibration reduced relative depth error from 27.4% to below 10%, while the door detection algorithm reached a precision of 0.93 and an F1‑score of 0.91. A dedicated ablation study shows that multi‑sensor fusion significantly improves stair‑recognition robustness compared to individual sensing modalities, and a failure‑case analysis delineates the limits of monocular perception under challenging lighting and reflective surfaces. The results demonstrate that reliable indoor exploration and stair traversal are achievable on resource‑constrained platforms without specialized ranging hardware, a practical, cost‑effective solution for rapid SAR deployment.
Authors:Chenming Zhu, Peizhou Cao, Jingli Lin, Wenbo Hu, Yunlong Ran, Jiangmiao Pang, Tai Wang, Xihui Liu
Abstract:
Human spatial understanding arises from jointly perceiving geometry and semantics, enabling consistent object identification and localization across viewpoints and time. Current video segmentation models depend on explicit object appearance memory banks for instance tracking, yet they remain vulnerable to large viewpoint changes and long‑term occlusions. Leveraging the spatial consistency afforded by modern feed‑forward 3D reconstruction models, we propose the Geometry Grounded Tracking Anything Model (G^2TAM), a unified framework for promptable instance tracking in 3D using only unordered RGB images or videos. G^2TAM employs spatially aligned geometric representations as implicit memory, ensuring stable instance identity and localization across frames and views. At its core is a cross‑modal spatial encoder that integrates visual and textual prompts into a shared geometric space, enabling end‑to‑end spatial reconstruction and instance‑consistent mask prediction. To support training and evaluation, we construct InsTrack, a large‑scale dataset with a dedicated validation split for benchmarking. Extensive experiments show that G^2TAM delivers strong cross‑view consistency, promptable instance spatial tracking, video object segmentation and spatial reconstruction, establishing a foundation for interactive, geometry‑grounded spatial reasoning.
Authors:Zhenfeng Su, Kang Zhao, Han Bao, Tao Yuan, Zhongzhe Hu, Xianzhi Yu, Wenxuan Wang
Abstract:
While prior studies have successfully compressed vision Transformers (ViTs) through various pruning techniques, most have concentrated on width pruning to achieve significant reductions in model size. Depth pruning, which removes entire layers from a ViT, is notoriously difficult for accuracy recovery despite its potential to deliver higher speedups, limiting the acceleration achieved by existing joint width‑and‑depth pruning methods. In this work, we reveal that the failure of existing depth pruning methods lies in their neglect of heterogeneity between different layers, and we introduce HetDPT, a heterogeneity‑aware depth pruning method that avoids dimension mismatch. Comprehensive experiments on ImageNet‑1K, CIFAR‑100, COCO, and ADE20K validate our method: HetDPT achieves a 1.58× speedup for DeiT‑B while maintaining accuracy and a 1.39× speedup for DeiT‑S with nearly no accuracy degradation. Furthermore, when combined with width pruning, HetDPT+ sets a new state‑of‑the‑art record in extreme ViT pruning, enhancing the acceleration ratio from 4.24× to 5.19× for the Isomorphic‑Pruning‑2.6G configuration while maintaining near‑lossless accuracy; our code is available at https://github.com/Efficient‑AI‑for‑All/HetDPT.
Authors:Liang Han, Wenyuan Zhang, Junsheng Zhou, Yu-Shen Liu, Zhizhong Han
Abstract:
Multi‑view 3D surface reconstruction is a longstanding challenge in computer vision. Although recent large‑scale reconstruction methods based on 3D Gaussian Splatting (3DGS) achieve impressive novel‑view synthesis, producing high‑quality surfaces over large scenes remains difficult, due to complex geometry, long optimization, and limited memory. In this paper, we propose a novel yet simple partitioning method to efficiently and faithfully reconstruct large‑scale scene surfaces. Our key insight lies in a scene partitioning method based on viewpoint orientation. This partitioning approach ensures that views with similar orientations are jointly involved for more accurate depth estimations, leading to precise surface reconstructions and balanced computation on multiple GPUs in parallel. In addition, we propose a strategy to detect and repair missing regions in the initial point cloud caused by sparse viewpoints or insufficient textures, thereby further improving the geometric quality. Extensive experiments on the GauU‑Scene, MatrixCity, and UrbanScene3D datasets demonstrate that our method outperforms the state‑of‑the‑art approaches in surface reconstruction for large‑scale scenes. Project page: https://hanl2010.github.io/VOP‑GS.
Authors:Yoshiro Sato
Abstract:
Classical training‑free denoisers such as BM3D and non‑local means owe much of their strength to search: content‑dependent block matching whose memory traffic and data‑dependent control flow parallelize poorly and preclude fixed‑latency implementations. Learned denoisers reach the highest quality, but they need training data, degrade outside their training domain (which we also observe), and carry per‑pixel compute budgets that effectively require a GPU. We present GALOSH (Generalized Anscombe LOcal SHrinkage), a redesign of training‑free denoising that removes the search entirely and aims at multi‑domain coverage, speed, and quality at once: a blind per‑image Poisson‑Gaussian noise fit, a generalized Anscombe transform, a two‑pass local Walsh‑Hadamard shrinkage of luminance, and a luminance‑guided local regression of chrominance ‑‑ two deliberately different operators for the two perceptually different noise components, each with its own strength control. Every stage is local, data‑independent, and regular ‑‑ the same computation graph for every pixel of every image. One core serves two domains: raw Bayer mosaics and sRGB/YUV images. On four real‑noise benchmarks (SIDD Medium and RawNIND, raw and sRGB) GALOSH is consistently the strongest among the tested blind, training‑free methods ‑‑ surpassing BM3D‑ and NLM‑family baselines even when those are given an oracle noise level ‑‑ and approaches trained networks on raw data while remaining below in‑domain trained networks at high ISO in sRGB. Being search‑free makes it fast: 7x‑650x faster than the DL baselines on the same GPU at full benchmark size, and the only strong method in the comparison that also runs practically on plain CPUs. The fixed, data‑independent structure is designed to map naturally onto fixed‑point and streaming hardware, supported by an operation‑count analysis and a working INT16 fixed‑point realization.
Authors:Liang Han, Bangcai Wei, Junsheng Zhou, Yu-Shen Liu, Zhizhong Han
Abstract:
3D reconstruction from sparse views is a challenging task in 3D computer vision. Recent studies on 3D Gaussian Splatting (3DGS) have achieved remarkable results with sparse views in novel view synthesis, yet reconstructing high‑quality geometric surfaces from sparse views remains a challenge, due to the limited geometry clues and the discreteness of Gaussians. In this paper, we propose a novel 3DGS‑based method for high‑fidelity surface reconstruction from sparse views. Our key insight is to introduce a normal‑guided depth propagation approach, which can extend depth information from high‑confidence regions to constrain the depth in low‑confidence areas. Additionally, we propose an abnormal depth edge‑aware regularization to address depth discontinuities caused by the discreteness of Gaussians. Extensive experiments on DTU and Tanks‑and‑Temples datasets demonstrate that our method outperforms the state‑of‑the‑art methods in sparse view surface reconstruction. Project page: https://hanl2010.github.io/DP‑GS.
Authors:Yiqing Wang, Maria A. Woodward, Ziyun Yang, N. Venkatesh Prajna, Chunming He, Leslie M. Niziol, Mercy Pawar, Ming-Chen Lu, Guillermo Amescua, Rachel Wozniak, Sejal Amin, Abinaya Krishnan, Prabhleen Kochar, Sina Farsiu
Abstract:
Microbial keratitis requires rapid pathogen identification to guide treatment, but culture‑ and PCR‑based diagnostics are slow and resource‑intensive. We developed a triple‑phase multimodal framework for bacterial‑versus‑fungal keratitis classification using slit‑lamp photographs acquired under blue‑light, sclerotic‑scatter, and white‑light illumination, together with clinical metadata. The model combines cross‑modality contrastive learning, modality‑specific fine‑tuning, and feature‑level multimodal ensemble learning for patient‑level prediction. We evaluated the framework on a multicenter dataset of 1,645 patients and 17,158 images from India and the United States. The model achieved 85.84% accuracy, 84.46% average F1‑score, and 0.885 AUC. Site‑specific evaluation showed that pooled results were overly optimistic, whereas resampling‑ and balance‑based re‑evaluation provided a more realistic assessment of cross‑site generalization. Under all settings, our framework remained the top‑performing approach. The code is available at https://github.com/yqwang01/TPMKA and dataset access will be provided subject to University of Michigan data‑sharing clearance.
Authors:Zanwei Zhou, Jiazhong Cen, Jiemin Fang, Yumeng He, Chen Yang, Sikuang Li, Fanpeng Meng, Zhikuan Bao, Wei Shen, Qi Tian
Abstract:
Precise control over complex dynamics remains challenging for modern video generative models, as text prompts alone often cannot specify physically plausible, fine‑grained motion and interactions. We introduce proxy‑conditioned video generation, where a coarse proxy video from physics‑based simulation or real‑world recording serves as a dynamics carrier to control foreground object motion. Given a proxy video and a text prompt, the goal is to synthesize a new video that preserves the proxy dynamics while generating novel content and plausible interactions aligned with the prompt. Since paired proxy‑target videos are difficult to obtain, we propose ProxyUp, a training‑free framework built on pretrained video generative models. ProxyUp first inverts the proxy video into an intermediate latent representation and applies region‑wise latent noising, preserving motion‑critical proxy latents while injecting noise into regions intended for text‑driven regeneration. To mitigate the distribution mismatch and weak foreground‑background coupling introduced by this heuristic latent composition, we further propose Stochastic Flow Relaxation (SFR), which progressively relaxes the composed latent toward the model's learned distribution before ODE sampling. Experiments on both simulation and real‑world proxies show that ProxyUp outperforms strong video editing and motion transfer baselines in dynamic fidelity and text alignment.
Authors:Kelin Yu, Haode Zhang, Harish Ravichandar, Yunhai Han, Ruohan Gao
Abstract:
Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact‑rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy‑agnostic real‑world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two‑stage design: it first bootstraps tactile‑aware learning from autonomous base‑policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact‑rich tasks, visual base policies, and tactile representations. Across four real‑world contact‑rich tasks, it improves visual base policies from 5‑40% success to 85‑100% within 40‑80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune‑site/
Authors:Weiyang Guo, Zesheng Shi, Longhui Zhang, Zeen Zhu, Min Zhang, Jing Li
Abstract:
Large language model (LLM) agents have shown strong decision‑making capabilities in long‑horizon interactive tasks, yet they still struggle to effectively leverage failed trajectories: full retries incur high interaction costs, while experience retrieval tends to dilute critical experience signals. To address this, we propose PivoARL, a self‑feedback retry framework for experience exploitation in LLM agents. PivoARL identifies the pivotal erroneous turn through structured reflection and performs local retry only from the corresponding pivotal state, thereby reusing the correct prefix and reducing redundant interactions. From an information‑gain perspective, we further show that pivotal retry concentrates useful experience signals near the error boundary, mitigating the signal dilution caused by state‑agnostic experience utilization. Based on this insight, we design a pivotal‑aware credit assignment mechanism that rewards correct prefixes while isolating erroneous suffixes, and optimize reflection quality through implicit reflection returns. We conduct a systematic evaluation on 4 agent tasks and 7 search‑based QA benchmarks. Results show that PivoARL achieves significant improvements on Pass@2/3 across all tasks, with an average gain of about 11.5% over MetaRL. Moreover, benefiting from contrastive preference signals induced by pivotal turns, PivoARL also consistently improves Pass@1 on over 80% of the tasks. On Minesweeper environment, PivoARL improves over GiGPO by more than 45% and reduces interaction turns by about 42% on average compared with full‑retry methods. Code is available at https://github.com/yuki‑younai/PivoARL.
Authors:Gongyang Li, Zhen Bai, Runmin Cong, Dan Zeng, Weisi Lin, Xiao-Ping Zhang
Abstract:
Existing Salient Object Detection in Optical Remote Sensing Image (ORSI‑SOD) methods mainly adopt the static inference strategy, which uses fixed trained model parameters for saliency inference in the testing phase. This means that even if the generated saliency map has errors, it cannot be further optimized. In this paper, we propose the novel IPDiff, a Diffusion‑driven ORSI‑SOD method with Information Reconstruction and Multi‑Prior Guidance. We build IPDiff based on a unique dynamic optimization strategy, which endows IPDiff with the ability to iteratively optimize saliency maps with a dynamic parameter. Specifically, we formulate ORSI‑SOD as a conditional diffusion problem in IPDiff. IPDiff first extracts informative conditional priors from ORSIs, including the saliency prior and the hierarchical priors, in the prior network with the assistance of the information reconstruction‑driven attention module. The saliency prior can provide positional information of salient objects, while the hierarchical priors can provide specific detail and semantic information of salient objects. Under the guidance of these priors, IPDiff then iteratively denoises random noise as the timestep dynamically changes in the denoising network, generating saliency maps that are close to ground truths. Notably, we simultaneously supervise IPDiff in both spatial and spectral domains through a hybrid loss function to achieve efficient network training. Comprehensive experiments on public ORSSD, EORSSD, and ORSI‑4199 datasets demonstrate that our proposed IPDiff achieves the best performance compared to 46 state‑of‑the‑art methods. The code and results of our method are available at https://github.com/MathLee/IPDiff.
Authors:Enshuo Hsu, Jin Zhou, Kirk Roberts
Abstract:
Extracting textual information from scanned medical documents, such as external laboratory reports and manually filled forms, has been a major challenge in modern electronic health records (EHRs). Recent advancements in vision language models (VLMs) have shown great promise over traditional OCR tools. However, at this point, most clinical OCR studies were conducted on private, institutional data. To our knowledge, there are few publicly available datasets for evaluating OCR models in the clinical domain. Furthermore, common scanning artifacts that undermine OCR performance are not reflected in those datasets, leaving a systematic evaluation unfeasible. Therefore, we release a publicly available, realistic‑looking OCR benchmark dataset, ClinOCR‑Bench, with 384 scanned images across 6 subsets: Normal, Handwriting, Poor Quality, Rotation, Tables, and Mix‑artifacts. ClinOCR‑Bench features: 1) diverse document types and layouts, 2) full coverage of common EHR scan artifacts, 3) protected health information‑free, 4) template‑aware train/test split, and 5) adequate sample size for OCR benchmarking. Baseline OCR performance was evaluated using state‑of‑the‑art open‑weight and proprietary VLMs. The dataset and documentation are available on GitHub (https://github.com/ClinOCR‑Bench/ClinOCR‑Bench).
Authors:Ayush Prasad, Swarnalee Mazumder
Abstract:
Decades of orbital missions have produced multi‑modal remote sensing data for the Moon, spanning optical imagery, spectroscopy, thermal emission, radar, gravity, and elemental composition. Yet these datasets remain fragmented across archives, and no benchmark exists for evaluating machine learning on lunar data. We introduce Moonstone, the first multi‑modal foundation model benchmark for lunar remote sensing. Our contributions are: (1) a 28‑channel, 128 pixels‑per‑degree (~237 m) global lunar pretraining dataset from seven instrument families across five missions, (2) MG‑MAE, a modality‑grouped masked autoencoder with per‑group convolutional tokenizers, a shared Vision Transformer encoder, attention masking for missing modalities, coverage‑adaptive masking for heterogeneous spatial coverage, and spectral continuity regularization for physically plausible reconstructions, and (3) a benchmark of six downstream tasks covering classification, regression, and segmentation. MG‑MAE pretrained features outperform scratch baselines on all tasks and surpass both ImageNet‑pretrained and vanilla MAE baselines by large margins. Data and code are available at https://huggingface.co/datasets/ayushprd/Moonstone and https://github.com/ayushprd/Moonstone .
Authors:Michał Mazuryk, Fleur Dolmans, Louis Gehringer, Ina Klaric, Jia-Huei Ju, Mohammad Aliannejadi
Abstract:
Recent work has suggested that adding irrelevant documents to the input of retrieval‑augmented generation (RAG) systems can improve question‑answering performance, a phenomenon referred to as the Power of Noise. This motivated investigations into the role of noise in information retrieval. In this paper, we reproduce the main findings of Cuconasu et al. and evaluate the robustness of the effect under extended experimental settings. We first confirm that the phenomenon holds under the original setup, which uses earlier‑generation LLMs, restrictive prompting and constrained decoding settings. We subsequently introduce a series of extensions to investigate the underlying causes of the noise effect, examining the authors' original design choices including the use of different models, instruction prompting, and relaxed output length constraints. Across these ablations, the Power‑of‑Noise pattern proves highly sensitive to inference configuration: it can appear, weaken, or disappear under small changes to prompt formulation and decoding limits. Combined with our error analysis, which shows substantial contributions from truncation and malformed generations, this variance indicates that the original effect cannot be robustly confirmed as a general benefit of noisy retrieval under these experimental conditions. More broadly, our work highlights the importance of carefully scrutinizing inference design in retrieval‑augmented generation systems. Our code is available at https://github.com/ina0105/The‑Power‑of‑Noise‑Reproduction.
Authors:Jianing Deng, Yuanzhe Li, Jialu Wang, Song Wang, Tianlong Chen, Huanrui Yang, Jingtong Hu
Abstract:
Feed‑forward 3D reconstruction (F3R) transformers have recently achieved remarkable success. However, scaling them to long image sequences remains challenging, as the quadratic complexity of cross‑view global attention quickly becomes the dominant computational bottleneck. While recent efforts attempt to improve efficiency through compressed or sparse attention, they fail to fully exploit the inherent sparsity and dynamic behavior of global attention. In this work, we present a comprehensive analysis of global attention across multiple F3R transformers and reveal that attention patterns are highly heterogeneous, dynamic, and extremely sparse across layers and attention heads. Motivated by these findings, we propose SAF3R, a training‑free dynamic sparse attention framework tailored to F3R transformers. SAF3R integrates tailored sparse attention mechanisms with offline head profiling and an efficient online adaptation strategy to match input‑dependent attention behaviors. Extensive experiments demonstrate that SAF3R achieves high sparsity ratios while preserving camera pose estimation and 3D reconstruction quality, translating into substantial end‑to‑end speedup on F3R transformers compared to existing methods. Code is available at https://github.com/jndeng/SAF3R
Authors:Dayou Mao, Yuchen Lin, Ashkan Ebadi, John Zelek, Alexander Wong, Yuhao Chen
Abstract:
Automation in construction is essential for reducing costs and human errors in large‑scale projects. We approach the construction progress monitoring from the aspect of detecting changes in construction sites. As construction buildings continue to evolve in geometry and appearance over time, change detection need to be performed from arbitrary camera viewpoints. This necessitates developing 2D Change Detection (2DCD) algorithms that operate robustly across diverse camera perspectives at construction sites. While developing and evaluating such systems is data‑intensive, no open‑source benchmark dataset exists at the intersection of 2D change detection and construction automation research. Data collection using Unmanned Aerial Vehicles (UAVs) is gaining its popularity in outdoor large‑scale surveying. However, in active construction sites conducting drone missions equipped with high‑end sensors imposes safety concerns. Flight trajectory and collected camera viewpoints can be significantly limited. To address this critical gap, we introduce iVISION‑2DCD, a large‑scale synthetically generated dataset from dense LiDAR point clouds with photorealistic input images and accurate ground truth annotations. Our dataset formally defines the problem of viewpoint‑robust 2DCD at construction sites and captures the inherent complexities of real‑world deployment. In this paper, we present our systematic methodology for synthetic data generation, developing novel view synthesis techniques to overcome bi‑temporal alignment and viewpoint diversity challenges, and implementing semi‑automated semantic segmentation with change label generation while preserving challenging real‑world cases. Benchmark evaluations using state‑of‑the‑art 2DCD algorithms demonstrate that iVISION‑2DCD poses novel research challenges for the computer vision and robotics communities.
Authors:Aron Asefaw, Konstantinos Tzevelekakis, Damian Falk, Léo Meynent, Damian Borth
Abstract:
Weight space learning aims to learn representations of neural network (NN) weights, enabling different downstream tasks. Existing approaches show promising performance, but lacking a way to shape these weight‑space representations using information about the datasets the models were trained on, thus limiting downstream applications. We propose WeightCLIP, a method for learning a dataset‑aligned latent space for neural networks, where datasets information is induced during training. The NNs are encoded as latent representations using an autoencoder, while dataset samples are encoded using a dataset encoder. The two representations are aligned using a contrastive objective, effectively reshaping the weight‑space representations according to the datasets. We demonstrate that such representations can be used for different downstream tasks, including mapping dataset information to a weight‑space representation that decode to strong models. In addition, we introduce a latent refinement process for generating models that outperforms standard fine‑tuning. Overall, our results demonstrate that explicitly incorporating dataset information improves what can be achieved with weight‑space representations across retrieval, generation, and refinement. Code will be available at https://github.com/HSG‑AIML/WeightCLIP.
Authors:Gaoxiang Luo, Yifan Wu, Sinian Zhang, Aryan Deshwal, Ju Sun
Abstract:
Large language models (LLMs) are increasingly deployed as critical decision‑making components in high‑stakes real‑world AI systems, rendering LLM reliability a foremost practical concern. In this paper, we focus on enhancing LLM reliability through selective prediction (SP), a strategy that allows an LLM to only predict for inputs where it is likely to be correct (i.e., coverage) and hence reduce the error rate (i.e., risk) on that portion of inputs ‑‑ flagging the remaining inputs for future human discretion. In other words, SP improves LLM reliability by balancing the risk‑coverage trade‑off and enabling seamless human‑AI collaboration. To integrate SP into LLMs, we focus on the LLM post‑training alignment stage and propose to align LLMs with SP performance metrics, in contrast with existing LLM alignment methods that focus primarily on correctness or calibration metrics. Specifically, we propose a novel alignment framework, Reinforcement Learning for Selection Reward (RLSR), which targets the area under the risk‑coverage curve (AURC) ‑‑ a popular SP performance metric ‑‑ as its alignment objective. RLSR achieves substantially better risk‑coverage trade‑off compared to multiple alignment baselines on both in‑domain and out‑of‑domain tasks.
Authors:Oren E. Livne
Abstract:
Graph‑based semi‑supervised learning (SSL) propagates a few labels over a similarity graph by minimizing a Dirichlet‑type energy. The standard quadratic (p=2) energy reduces to a single graph‑Laplacian solve, but it degenerates exactly where SSL is most useful when labels are scarce: gathering more unlabeled data drives the p=2 estimate to a near‑constant function whenever d\ge2 (Nadler‑Srebro‑Zhou). Well‑posedness requires the nonlinear p‑Laplacian energy with p>d. Existing solvers reduce this to a sequence of weighted Laplacian solves, but their reference implementations use a direct sparse factorization or ichol‑preconditioned CG instead. Plugging a near‑linear Laplacian solver is not straightforward: at large p the conductance weights degenerate near flat‑gradient edges, making the system nearly singular and causing stagnation without a damped outer iteration. We close this gap. Recasting p‑Laplacian SSL as a source‑form nonlinear Laplacian flow Bρ_p(B^\top x)=b and solving by damped chord‑Newton continuation in p, every linearized system stays well‑conditioned and can be delegated to a near‑linear Laplacian engine. On size‑scaled graph families the wall‑clock is empirically m^0.96‑m^1.02 per family (approximate Cholesky default), and a pooled fit across 228 SuiteSparse graphs gives m^1.19 vs.\ m^1.45 for direct factorization; the solver handles a 6.8×10^7‑edge social network in minutes. Memory is the binding constraint: Cholesky fill reaches 10‑280× the graph nonzeros vs.\ our O(m) hierarchy. Against the released FCL solver we are 1.5‑14× faster at matched accuracy. On MNIST 10‑NN, p=3 scores 64% at one label per class vs.\ 36% for p=2. Code: https://github.com/orenlivne/np.
Authors:Bach-Hoang Ngo, Si-Tri Ngo, Hieu Le, Trung-Nghia Le
Abstract:
Text‑to‑image diffusion models fail to generate correct object counts in dense scenes, where overlapping instances collapse into indistinguishable structures despite appearing visually plausible. We identify this as instance ownership collapse: tokens from overlapping objects interact freely through attention, while heavily occluded instances receive weak supervision due to their small visible areas. We address this through layout‑aware attention biases that softly bias token interactions toward region‑consistent grouping and suppress cross‑instance leakage, paired with an amodal‑balanced loss that amplifies gradients for occluded objects based on their occlusion level. To enable systematic evaluation, we introduce OverlapDepth‑45K, a benchmark of densely overlapping scenes with amodal supervision. Our approach substantially improves count accuracy and prevents instance merging while preserving image quality. Project page: https://bachngoh.github.io/AIBL
Authors:Alberto Foresti, Ivan Butakov, Alexander Tolmachev, Giulio Franzese, Alexey Frolov, Pietro Michiardi
Abstract:
Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low‑dimensional distributions, leaving their performance on complex, realistic data largely unexplored. We address this gap with a comprehensive benchmarking framework grounded in a unified copula‑theoretic perspective that subsumes existing benchmarks as special cases. Within this framework, we propose two complementary families of tests: a copula‑first family that systematically varies ground‑truth MI, dimensionality, and marginal complexity using synthetic and flow‑based transformations; and a marginals‑first family that couples real‑world image data with controlled dependency structures, extending the classic same‑class‑pairing paradigm. We use this suite to extensively evaluate three classes of estimators: non‑parametric, discriminative, and generative. Contrary to prevailing assumptions, our results indicate that there is no universal winner: each category can systematically outperform all other estimators under specific setups. By analyzing these cases, we identify fundamental estimation barriers and propose new tests that more effectively stress these specific limitations. We share the open source code at https://github.com/VanessB/mutinfo.
Authors:Xuan-Bach Mai, Duy-Phuc Nguyen, Quoc-Van Le, Tam V. Nguyen, Thanh-Toan Do, Huu Le, Duong-Van Nguyen, Minh-Triet Tran, Trung-Nghia Le
Abstract:
Synthesizing physically accurate mirror reflections remains a fundamental challenge for modern text‑to‑image diffusion models, which are increasingly critical for generating synthetic training data for embodied AI and robotic perception. These models typically struggle with strict geometric constraints, leading to hallucinations that degrade the utility of the synthetic data. To address this, we introduce a novel, end‑to‑end physics‑aware generation framework namely PhysMirror that natively enforces projective geometry through explicit 3D spatial priors. Our method automatically lifts prompted objects into 3D meshes and constructs a lightweight, mathematically exact mirror scene within a simulated environment. By rendering this explicit 3D scene, we extract precise 2D conditioning elements, such as depth maps and segmentation maps, that serve as robust guiding signals for downstream diffusion models, guiding them to generate images with physically correct mirror reflections. Moreover, we introduce Mirror Consistency Score (MCS), reference‑free, fully automated metric that quantifies physical correctness using dense feature matching and vanishing point convergence. Experimental results on our newly constructed MirrOB dataset demonstrate that our approach outperforms state‑of‑the‑art baselines in reflection accuracy and physical realism, while maintaining strong text‑to‑image semantic alignment, providing a reliable pipeline for embodied AI data generation. The source code is released at https://duyphuc0701.github.io/PhysMirror.
Authors:Yifei Shen, Bo Li, Xinjie Zhang
Abstract:
While skill optimization for autonomous agents has gained traction, existing methods rely on complex pipelines. This leaves a fundamental question unaddressed: What constitutes a minimal viable pipeline for skill optimization, where every component is justified by theory or empirical necessity? We formalize skill optimization via Zeroth‑Order (ZO) optimization, mapping classical counterparts (central difference, trust regions) to recent literature. Noting that unlike blind numerical perturbations in classical ZO, skill trajectories serve as interpretable debugging feedback. Grounded in Claude Code philosophy and PAC learning, we establish three principles for convergence and generalization: file‑system‑based trajectory exploration, consensus attribute mining, and independent validation gating. Eliminating redundancies, we propose SkillOpt‑Lite. It accelerates convergence and outperforms full SkillOpt: improving LiveMath by +8.8 points on GPT‑5.5 and +25.4 points on GPT‑5.4‑nano, allowing the nano model to surpass standard GPT‑5.4 optimized by SkillOpt. Finally, we integrate our framework into production coding agents like VSCode Copilot, enabling developers to evolve agent skills via one line of vibe. Because our framework treats all agent components simply as standard editable code, this minimal pipeline naturally generalizes to full harness optimization (HarnessOpt). On SpreadsheetBench, HarnessOpt enables GPT‑5.4‑nano to achieve 0.7758 accuracy, outperforming the larger GPT‑5.5 running standard pipelines (0.7620). Code is available at https://github.com/EvolvingLMMs‑Lab/SkillOpt‑Lite.
Authors:Pengwei Zhang, Bin Xie, Ce Hao, Xinpan Meng, Xinyu Guo, Fang Deng, Long Cheng, Tiancai Wang
Abstract:
Tactile perception is indispensable for contact‑rich manipulation, yet integrating it into Vision‑Language‑Action (VLA) models often induces modality collapse, where high‑bandwidth visual features overshadow sparse tactile cues. Inspired by Predictive Coding, a neural mechanism where the brain attenuates predictable inputs to prioritize surprising stimuli, we propose ResTacVLA. Rather than treating tactile data as raw input, we reformulate it as a Residual Tactile Representation capturing the discrepancy between visual priors and physical sensations. By filtering out visually predictable dynamics, this formulation transforms sparse tactile signals into dense, high‑value information gain, thereby inherently resolving the bandwidth mismatch. These residuals are discretized through a Vector Quantized (VQ) bottleneck into Latent Contact Primitives that capture critical events missed by vision. Analogous to the neural surprise signal, we leverage the uncertainty of the visual prior to adaptively gate tactile integration, prioritizing residuals specifically during visually unreliable phases to explicitly prevent visual dominance. Experimental results show that ResTacVLA consistently outperforms all baselines on a diverse set of contact‑rich manipulation tasks, while remaining robust to unexpected dynamic disturbances. Project page: https://awilekong.github.io/ResTacVLA/
Authors:Huajun Bai, Weiwei Lv, Huichuan Zheng, Youyou Lu, Jiwu Shu
Abstract:
LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought‑Action‑Observation loop is often serial: the model reasons, emits a tool call, then idles the GPU until the result returns. This wait consumes 16‑37% of wall time in our workloads and 35‑61% in prior reports. Speculative tool execution can hide this wait, but existing systems need auxiliary predictors, historical traces, or static workflow graphs, leaving a gap for training‑free, day‑one deployment. We observe that the model can be its own predictor: a probe forked at the start of generation predicts Qwen3‑32B's upcoming tool name with 74.6‑99.6% accuracy across five benchmarks. We present SPORK (Self‑sPeculative fORKing), a training‑free controller that dispatches the speculated tool call early, overlapping its execution with the remaining chain‑of‑thought decode. A cost model captures when speculation breaks even, and each component improves one of its terms: a prefix‑cache fork cuts probe cost, a confidence gate filters mispredictions, and partial‑token accept turns rejected probes into speculative‑decoding drafts. On acceptance, the tool result is ready when reasoning ends; on rejection, SPORK falls back to serial execution with no correctness penalty. On real‑tool benchmarks, SPORK cuts Qwen3‑32B's GAIA P95 by 18% (131.9 to 108.1 s); the mechanism holds across model sizes from 4B to 32B and across dense and mixture‑of‑experts models, with task accuracy within 1 pp of baseline or better wherever measured. SPORK deploys as a thin controller over standard completion APIs (no retraining, no auxiliary models, no offline traces) and is orthogonal to token‑level speculative decoding. SPORK is open source at https://github.com/baihuajun24/spork.
Authors:Joaquin Gajardo, Michele Volpi, Marko Mihajlovic, Siyu Tang, Lukas Roth, Sergey Prokudin
Abstract:
Quantifying plant growth dynamics from sparse longitudinal 3D observations is fundamental for agriculture and plant sciences. Yet, plants pose unique challenges: they undergo intricate non‑rigid deformations, exhibit changing topology as new organs emerge, and often lack explicit temporal correspondences between consecutive data acquisitions due to newly formed tissue. Methods designed for general scenes struggle to model topology changes and asynchronous organ growth characteristic of plants. To address these challenges, we introduce GrowFields, a compositional dynamic neural field representation for organ‑aware 4D plant growth modelling from point cloud time series. Our approach decomposes a plant into its constituent organs and aligns each organ into its own canonical coordinate frame, isolating intrinsic growth patterns from global plant motion. We then learn a shared continuous neural deformation field that models temporal dynamics across all organs, conditioned on learnable per‑organ latent codes capturing organ identity and growth characteristics. The resulting modular yet unified representation naturally accommodates the asynchronous development of plant organs while remaining grounded in the practical setting of organ‑level plant tracking. We evaluate GrowFields on growth sequences from four plant species, assessing geometric fitting and organ tracking accuracy using manually annotated leaf‑tip trajectories. Results demonstrate consistent improvements in spatial precision, temporal coherence, and morphological fidelity over a range of existing representations.
Authors:Xinze Liu, Ding Wang, Hengjie Zhu, Dayan Wu
Abstract:
Efficient large‑scale image retrieval requires compact representations that preserve semantic similarity under fast Hamming‑space search. Deep hashing is appealing, but most existing CNN‑ and ViT‑based methods still follow a post‑quantization paradigm, where continuous visual features are first learned and binary codes are then produced by a terminal hash projection or binarization operation. This late code generation creates a feature‑to‑code discrepancy between the continuously optimized representation space and the discrete Hamming space used for retrieval. To address this limitation, we propose HashViT, a Vision Transformer framework for native hash token learning. Instead of treating hashing as a terminal readout, HashViT introduces a dedicated HASH token that serves as a persistent, hash‑oriented retrieval state inside the transformer. The HASH token is structurally decomposed into a Hash Register for direct binary code generation and a Semantic Workspace for preserving auxiliary continuous semantics. To enable effective workspace‑to‑register interaction, we further design a lightweight Hash Refinement Adapter that progressively refines the Hash Register across transformer layers. As a result, binary‑oriented representations are formed through token evolution within the backbone, rather than being abruptly induced by an output‑level projection. HashViT is optimized with a unified objective that combines learnable semantic center supervision, class‑token similarity distillation, and quantization regularization, encouraging the HASH token to encode semantically structured and compact binary representations. Extensive experiments on three widely used benchmarks demonstrate that HashViT achieves state‑of‑the‑art or highly competitive retrieval performance while preserving the efficiency of compact Hamming codes. Code is available at https://github.com/Xinze919/HashViT.
Authors:Florian Fürnrohr, Reinhard Heckel
Abstract:
Real‑time cardiac cine MRI enables visualization of the beating heart during free breathing, but severe undersampling and motion make reconstruction highly challenging. A central challenge for reconstruction is incorporating powerful priors of cardiac anatomy while remaining computationally efficient. We propose Piecewise Dynamic Diffusion Regularization (PDDR), a reconstruction method that integrates a spatiotemporal diffusion model as a generative prior within a variational reconstruction framework for cine MRI. The model employs dedicated spatial layers to encode anatomical structure and temporal layers to capture cardiac motion learned from gated cine data. PDDR leverages the dynamic prior in a piecewise manner, enabling the efficient use of spatiotemporal diffusion models for processing of long real‑time sequences. Experiments on retrospectively accelerated and prospective real‑time cine MRI demonstrate that PDDR outperforms classical, unsupervised, and diffusion‑based methods, delivering high‑quality reconstructions with substantially reduced computation time compared to state‑of‑the‑art baselines. These results highlight PDDR as a practical and scalable solution for free‑breathing, real‑time cardiac MRI. Code is available at https://github.com/MLI‑lab/pddr.
Authors:Jialiang Wang, Xianming Liu, Xiong Zhou, Hui Liu, Haoliang Li
Abstract:
The alignment of large language models with human preferences is commonly achieved through Reinforcement Learning from Human Feedback or Direct Preference Optimization. However, these methods are vulnerable to the significant noise prevalent in real‑world preference datasets. To address this critical issue, we present a theoretical framework for unbiased alignment, introducing the Unbiased Reward Model (URM) loss and the Unbiased Direct Preference Optimization (UDPO) loss. By mathematically correcting the distortion induced by preference noise, our novel objectives enable unbiased model training directly from noisy datasets, without requiring clean ground‑truth supervision. We provide rigorous theoretical analyses demonstrating that our methods are noise‑tolerant, parameter downward compatible, and classification‑calibrated. Comprehensive experiments across diverse datasets demonstrate that our approaches outperform state‑of‑the‑art baselines. Code available at: https://github.com/cswjl/unbiased‑alignment.
Authors:Chengcheng Wang, Tingzhang Luo, Wenhao Li, Jianyuan Guo, Chang Xu
Abstract:
Diffusion language models (DLLMs) generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this trajectory and commit tokens from the current snapshot alone, conflating confidence with commitment readiness: a transient top‑1 peak under incomplete context can be locked in, while candidates with consistent cross‑step support are delayed. We propose Trajectory‑Aware Commit Gating (TACG), a training‑free gate‑level decoder that anchors token identities to the base posterior and uses trajectory‑aware signals only to decide whether the current proposal is ready to commit. TACG combines Temporal Implicit Logits Guidance (TILG), which keeps an exponential moving average of past logits as a self‑reference and contrasts the current logits against this reference in natural‑parameter space, with a History Gate (HG) that enforces short‑term proposal persistence before commitment. Together with a capped extra‑promotion budget, these components yield a stability‑constrained commit rule without auxiliary networks or extra forward passes. We evaluate TACG on LLaDA, Dream, and LLaDA2‑Mini across code (HumanEval, MBPP) and math (GSM8K, MATH500) benchmarks; it typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward (TPF). The code is publicly available at https://github.com/Clarence‑CV/TACG‑DLLM.
Authors:Geng Li, Yuxin Peng
Abstract:
While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine‑grained perception in ultra‑high‑resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior‑free scanning, or depend on static prior‑driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial‑scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early‑stop attention rollout of MLLM to construct reasoning‑aware priors, while employing a scale‑aware non‑stationary kernel and GP‑UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub‑linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state‑of‑the‑art baselines with a superior trade‑off between accuracy and efficiency.
Authors:Zheng Sun, Lerong Zhang, Zhihao Li, Zhuo Li, Quentin Rouxel, Fei Chen
Abstract:
Generalizable robot manipulation requires stable 3D understanding of functional object parts, such as handles, tool heads, openings, and graspable regions. Raw point clouds provide geometry but lack explicit part semantics, and their sampled points vary with viewpoint, sensor configuration, and object instance. Existing 2D feature lifting and discrete 3D point‑wise features enrich point clouds with semantics, but the resulting features remain attached to observation‑dependent samples. We propose an object‑centric continuous semantic field that conditions on an object point cloud and reads part‑aware semantic embeddings at explicit 3D query locations. The field is trained from part‑annotated object models and then frozen to generate semantic point clouds as object‑level conditioning for manipulation policies. Experiments on RoboTwin simulation tasks and real‑world bimanual object manipulation show that our representation provides more stable functional‑part cues and improves policy performance over raw point‑cloud, 2D feature lifting, and 3D point‑wise feature baselines. Project Page: \hrefhttps://zainzh.github.io/beyond‑point‑attached‑semanticshttps://zainzh.github.io/beyond‑point‑attached‑semantics.
Authors:Estera Dumitru, Stelian Spînu
Abstract:
Modern video surveillance systems generate far more video streams than human operators can effectively monitor, making automated analysis essential for timely detection of security events. This paper presents a unified multi‑task deep learning framework that simultaneously performs face recognition with zone‑based authorization, automatic license plate recognition, weapon detection, fire and smoke detection, and human action recognition on a shared GPU platform. Among the integrated modules, two task‑specific deep‑learning models are proposed in this work to address scenarios that are insufficiently represented in publicly available datasets: a single‑class weapon detector fine‑tuned on a merged and relabeled dataset, achieving a mean average precision (mAP@0.5) of 0.947, and a SlowFast‑R50 action recognition model trained on a purpose‑built vandalism dataset comprising 614 video clips, achieving 94.33% classification accuracy. To improve robustness in continuous video, all detection modules are integrated into a temporal event‑validation architecture based on multi‑frame confirmation, confidence‑weighted voting, and cascaded filtering, transforming frame‑level predictions into reliable security events. Each module is evaluated independently on established public datasets (LFW, D‑Fire, FIRESENSE, and UCF‑Crime), followed by integrated end‑to‑end system evaluation. The proposed temporal validation strategy reduces the fire and smoke false‑alarm rate from 52% to 4% and improves video license plate exact‑match accuracy from 66.7% to 81.8%, while the complete framework maintains real‑time operation with a per‑frame latency below 100 ms on commodity hardware. These results demonstrate that combining specialized deep‑learning models with temporal event validation provides an effective and practical solution for reliable real‑time intelligent video surveillance.
Authors:Yao Liu, Lishen Qu, Shihao Zhou, Jie Liang, Hui Zeng, Yabin Peng, Huipeng Lin, Lei Zhang, Jufeng Yang
Abstract:
Multi‑Exposure Fusion (MEF) effectively extends dynamic range, but practical deployment is hindered by motion‑induced ghosting and the scarcity of high‑quality dynamic benchmarks. Current benchmarks largely neglect dynamic scenes and lack reliable ground truth, making it difficult to handle the complexity of real‑world motions. In response, we introduce ExpoMotion, a large‑scale benchmark designed to evaluate deghosting capabilities. Comprising 1,738 sequences and 10,909 images across diverse environments, it covers a wide range of motions and provides high‑fidelity GTs constructed through an expert‑guided acquisition pipeline. To tackle the complex dynamics and extreme conditions captured in this benchmark, we propose the Householder Orthogonal Projection network (HOP), which revisits MEF deghosting from a mathematical perspective via Householder transformation, decoupling multi‑frame alignment into exposure pre‑alignment and ghost filtering. Specifically, the Global Priors Illumination Alignment (GPIA) module first rectifies drastic dynamic range discrepancies by utilizing global statistics for exposure harmonization. Regarding ghost removal, our Householder Orthogonal Attention (HOA) models artifacts as orthogonal perturbations. By employing a dynamic Householder reflector, HOA effectively projects ghosts out of the feature manifold while preserving high‑frequency details. Experiments demonstrate that our ExpoMotion dataset enables superior generalization and artifact‑free detail restoration, while also validating the effectiveness and efficiency of the HOP method. The dataset and code are available at https://github.com/Leo‑LiuYao/ExpoMotion.
Authors:Nicolas Sournac, Ahmed Baha Ben Jmaa, Bertrand Braeckeveldt
Abstract:
Safety‑critical applications require classifiers that are both robust and reliable. Adversarial training is a widely adopted defense for improving robustness in deep neural networks; however, its effect on the reliability of predictive uncertainty remains underexplored. We investigate this gap through the lens of selective classification, which has rarely been systematically analyzed alongside adversarial robustness. We introduce a unified benchmark for the robustness‑uncertainty trade‑off. It standardizes architectures, augmentations, threat models, and evaluation metrics across clean, adversarial, and common‑corruption settings. Across a wide range of state‑of‑the‑art adversarial training methods, we uncover a recurring failure mode: several approaches improve robust accuracy while degrading uncertainty ranking, leading to poorer selective behavior. To address this, we propose Evidential Adversarial Training (EV‑AT), which models uncertainty through a Dirichlet distribution and combines (i) an evidence‑based loss promoting clean accuracy and reliable uncertainty with (ii) a robust evidence‑alignment loss matching clean and adversarial predictions in log Dirichlet‑parameter space. Extensive experiments show that EV‑AT shifts the Pareto frontier of robustness‑uncertainty trade‑offs beyond prior state‑of‑the‑art adversarial training methods. Our source code is publicly available at https://github.com/NicolasSournac/Robustness_Meets_Uncertainty.EV‑AT.
Authors:Wenlin Wu, Sheng Zhou, Peipei Song, Wenhao Wang, Junbin Xiao, Xun Yang
Abstract:
As video generation paradigms evolve from localized manipulation to full‑scene synthesis, AI‑generated video detection becomes increasingly challenging, as forgeries exhibit coherent global structure and high perceptual realism. However, existing benchmarks are biased toward perceptual fidelity and primarily evaluate detectors based on perceptual artifacts, providing limited coverage of scenarios that require reasoning about violations of physical laws, structural coherence, or social logic. This dataset bias shapes current approaches and results in a Perception‑Reasoning Gap: artifact‑centric models capture low‑level statistical irregularities yet lack semantic inference, whereas vision‑language models perform semantic reasoning but remain insensitive to fine‑grained forensic cues. To bridge this gap, we propose SafeGuard, a multi‑agent framework that enables collaborative specialization between forensic perception and semantic reasoning. A hierarchical perceptual solver extracts fine‑grained forensic evidence, while a self‑reflective verifier enforces consistency between semantic inference and physical plausibility, forming an interpretable evidence chain. To support evaluation, we introduce SafeVid, a novel AI‑generated video detection benchmark comprising 20K videos spanning 10 social risk categories, designed to evaluate physical plausibility, structural consistency, and the rationality of social behaviors. Extensive experiments demonstrate the generalization of SafeGuard, improving accuracy on SafeVid by +18.7% and consistently outperforming prior methods across four public benchmarks.
Authors:Jiachi Zhang, Zhuoyu Wu, Quanjun Wang, Wenhui Ou, Wenqi Fang
Abstract:
While lightweight polyp segmentation is highly desirable for low‑cost deployment, reported performance gains often stem from upgraded backbone encoders, complex decoders, or heavy refinement branches. Consequently, it remains difficult to isolate whether a lightweight correction mechanism is inherently effective on its own. We address this limitation by formulating refinement as a prediction‑space recursive correction task, introducing a recursive controller that operates directly on backbone logits. Under a fixed recursion budget, this controller aggregates discrepancy and uncertainty evidence, updates a compact state tracking recent correction utility, and applies additive residual logit corrections. By design, this correction path remains small, host‑portable, and deployment‑explicit. Utilizing a unified Kvasir‑trained protocol, we evaluate our approach across seven lightweight backbones on Kvasir‑SEG and three transfer datasets, measuring segmentation accuracy (Dice/IoU) alongside deployment efficiency (parameters, GMACs, and peak memory). The controller yields consistent improvements in the source domain, achieves competitive performance against both training‑side baselines and heavier structural refiners on representative hosts, and delivers selective transfer gains with minimal static overhead. Code is available at https://github.com/tyui99/Gain‑Aware‑Prediction‑Space‑Recursive‑Controller.
Authors:Jiachi Zhang, Zhuoyu Wu, Wenqi Fang
Abstract:
Post‑refinement can improve colonoscopy segmentation after host inference, but many designs still rely on extra correction heads or multi‑stage pipelines with non‑negligible parameter or computational cost. For polyp segmentation, host predictions are often already reasonable globally, with remaining errors clustered around ambiguous boundaries and difficult local structures. These residual errors matter in colonoscopy images because useful masks need correct lesion coverage and clean contour delineation across subtle mucosal transitions. This setting favors selective local repair in prediction space over reprocessing the entire mask. We therefore propose RIGS‑Refiner, a lightweight post‑refinement plugin for risk‑guided recursive refinement in prediction space. Starting from a frozen host anchor prediction, RIGS‑Refiner extracts lightweight image priors and prediction cues, applies risk‑guided update, and writes back residual corrections through a shared recursive cell. The module adds only +519 parameters and +0.631 GFLOPs, keeping the refinement path compact for deployment. Experiments use Kvasir‑SEG for training and Kvasir, ClinicDB, ColonDB, and ETIS for evaluation under two frozen hosts, namely PraNet and SegFormer‑B0. Results show consistent gains on both hosts and a favorable efficiency‑accuracy trade‑off against representative post‑refinement methods. Code is available at https://github.com/tyui99/RIGS‑Refiner.
Authors:Boseong Kim, Donghyeon Cho
Abstract:
The presence of composite degradations poses a significant challenge, since the underlying corruption factors exhibit complex and interdependent interactions. Even when the degradation types are known, accurately restoring the image remains difficult due to the intertwined nature of their effects and the need for selective control during the recovery process. To address this, we introduce CURE, a unified framework that enables controllable restoration in complex degradation settings by learning disentangled and adjustable representations. CURE is driven by four complementary objectives. First, an identity embedding is incorporated, along with a reconstruction constraint, to ensure that the model can reproduce the input image when restoration is unnecessary. Second, the ratio control mechanism blends the identity embedding with degradation‑specific embeddings using user‑regulated mixing ratios, allowing continuous control over restoration intensity. Third, an intermediate loss is applied to supervise stepwise outputs, each encouraged to tackle the removal of only a single degradation factor within a composite mixture. Finally, a permutation‑invariant loss ensures that the model achieves consistent restoration quality regardless of the order in which multiple degradations are addressed. Since CURE modifies only the training strategy and not the underlying network architecture, it can be seamlessly integrated into existing controllable restoration models. Experiments demonstrate that CURE delivers state‑of‑the‑art performance on composite degradation benchmarks, while enabling both selective and jointly fused restoration through flexible modulation of embedding ratios. The code and dataset are available at https://github.com/bo‑oseng/CURE.
Authors:Chaesong Park, Jihyeon Hwang, Muyeol Sung, Jongwoo Lim
Abstract:
Omnidirectional depth estimation from multi‑fisheye camera rigs is complicated by visibility conflicts: wide baselines cause different cameras to observe different portions, or even different faces, of the same object, so aggregating their features into a unified equirectangular (ERP) representation under fixed projection produces ambiguous matching evidence near occlusion boundaries and thin structures. Although existing methods mitigate this by down‑weighting unreliable views, they do not resolve the underlying discrepancy because context formation and cross‑view fusion remain tied to rigid fisheye‑to‑ERP sampling. We present OmniDS, an iterative depth refinement framework that replaces rigid aggregation by combining dynamic context fusion with consensus‑aware multi‑view similarity. A dual‑stream encoder pairs a lightweight CNN for geometric detail with a frozen DINOv3 for semantic priors; their features are reprojected into ERP space at each refinement step via learned view weighting and deformable cross‑attention with geometric distortion bias. In parallel, a multi‑view consensus volume captures global cross‑camera agreement through group‑wise correlation and feature variance, regularized by a 3D U‑Net. For efficient deployment, we distill the dual‑stream representation into a single MobileNet‑based encoder. OmniDS achieves state‑of‑the‑art performance on the OmniThings, OmniHouse, and Sunny benchmarks while maintaining competitive inference speed. Project page and codes are available at https://parkchaesong.github.io/omnids.
Authors:Yaodong Su, Hanchang Li, Quanqing Xu, Chuanhui Yang, Yixiang Fang
Abstract:
In emerging systems (e.g., social media and e‑commerce platforms), data records are often drawn from heterogeneous sources, such as relational tables, text documents, image repositories, spatial databases, and knowledge graphs. Accordingly, retrieving target records for question‑answering (QA) tasks requires us to jointly exploit these heterogeneous sources. However, most existing benchmarks are constructed from individual sources, and only a very few recent benchmarks have considered two or three sources. To alleviate this issue, we introduce HETERQA, a comprehensive benchmark with 857 QA pairs for record retrieval over five heterogeneous sources. HETERQA instantiates this setting with Yelp business records, each of which is grounded by multiple sources. We build HETERQA in an answer‑driven manner: candidate records are first initialized with record‑field constraints, then enriched through heterogeneous sources, and finally cross‑verified across required sources before the natural‑language question is retained. We validate the benchmark through contradiction detection and human validation, and further evaluate sparse, dense, hybrid, late‑interaction, and agentic retrievers under the same metrics. The results show that HETERQA is challenging: hybrid retrieval achieves the strongest Recall@10, Self‑RAG achieves the best MRR@10, and all evaluated methods remain far from saturating the benchmark. These findings indicate that HETERQA provides an effective testbed for record retrieval over heterogeneous sources and leaves substantial room for future retrieval methods. The benchmark dataset and source code are publicly available at https://huggingface.co/datasets/hanchang02/HeterQA and https://github.com/hanchang02/HeterQA, respectively.
Authors:Yishu Wang, Yuxuan Wang, Jiaqi Deng, Hanyang Tang
Abstract:
Forecasting future events has attracted growing attention as a testbed for general‑purpose AI. A natural way to ground this evaluation is let the models trade in the prediction markets. Trading, however, requires more than forecasting. Moreover, recent benchmarks report a substantial gap between calibrated probability scores and the trading results. We propose Raven‑Agent, to the best of our knowledge, the first autonomous trading agent for prediction markets. On a controlled replay over an archived decision set, our architecture achieves the only positive return and the only positive risk‑adjusted return among all tested policies. We have released our code in https://github.com/Alchemist‑X/predict‑raven .
Authors:Wanshu Fan, Xiangyu Li, Cong Wang, Kin-man Lam, Xin Yang, Haiyan Zhang, Dongsheng Zhou
Abstract:
Images captured by consumer electronic devices, such as mobile phones and digital cameras, often suffer from low‑light degradation due to sensor limitations and imaging pipelines, which degrades visual quality and affects downstream vision tasks. Existing methods based on Convolutional Neural Networks (CNNs) and Transformers have dominated current low‑light image enhancement (LIE) due to their excellent ability to model hierarchical features. However, CNNs operate in local receptive fields that cannot model long‑range dependencies, while Transformers overcome this problem but incur substantial computational costs. To address these challenges, we propose MambaLIE, a Scene Light Intensity‑Boosted Low‑Light Image Enhancement method based on a State Space Model (SSM). We first introduce scene light intensity to improve the structural distribution of illumination, which is then gated with the low‑light input to guide enhancement. To better model the illumination while maintaining computational efficiency, we propose the Locally Enhanced State Space Model (LESSM) for efficient light enhancement. Our LESSM contains two branches: an SSM branch and a Local Enhanced branch, where the former is used to model the long‑range dependencies with linear time complexity, while the latter is used to enhance local feature representations. Extensive experiments demonstrate that MambaLIE outperforms state‑of‑the‑art CNN‑based and Transformer‑based LIE methods on four widely used synthetic benchmarks and five publicly available real‑world benchmarks in terms of accuracy, speed, and model size, making it suitable for practical deployment on resource‑constrained devices.
Authors:Igor Buyanov, Nafisa Valieva, Ekaterina Mazurina
Abstract:
Social media posts are a rich and valuable source of data for analyzing mental health states and users' well‑being using automated analysis tools. In this work, we demonstrate how we used a range of Natural Language Processing (NLP) methods, including Long Short‑Term Memory (LSTM), BERT‑based models, and Large Language Models (LLMs), for self‑state and well‑being analysis and summarization during the CLPsych Shared Task 2026. Our approach achieved one of the top Consistency and Contradiction scores for the summarization task and also middle‑level results for the other tasks. By testing and developing such mental health‑state estimation systems, we contributed to improving mental health support systems. We make our code available https://github.com/psytechlab/CLPsych2026/.
Authors:Sirong Pan, Guannan Tian, Pan Song
Abstract:
This paper contributes to vehicle dynamics modeling by introducing a physics‑informed neural state‑space model tailored for the parking regime of a production battery‑electric sedan, identified entirely from field‑test maneuvers. At parking speeds the model captures what the kinematic idealization omits, including actuator lag, drivetrain creep, brake‑hold transitions through standstill, and frequent reversals of the motion direction. A gear‑conditioned velocity constraint is imposed during training, and the yaw rate is read out as a learned residual on a kinematic‑bicycle prior, so that the network devotes its capacity to the deviation from physics rather than to its reproduction. These training‑time physics make the customary inference‑time state limiter redundant. The commanded‑to‑actual behavior of the drive, brake, and steering actuators is reproduced by dedicated submodels, for which signal fidelity proves an unreliable proxy for closed‑loop value; tuning the brake on its velocity consequence rather than on its own signal reverses the verdict reached at the signal level. The model generalizes to held‑out maneuvers in fully open‑loop simulation, and, despite being identified from only 16 field tests, the assembled command‑to‑vehicle chain earns Good ratings on the vehicle states under the ISO/TS 18571 objective rating metric. Embedded as the real‑time plant of an interactive simulator, it enables a production‑representative planning stack to park the vehicle through the learned dynamics. This makes the model suitable for pre‑calibrating an automated‑parking planning and control stack in the virtual development phase without the manufacturer's proprietary chassis and actuator parameters.
Authors:Fang Liu, Jinpeng Chen, Ke Xu, Yuhao Liu, Huankang Guan, Xudong Lu, Bo Yang, Gerhard Hancke, Rui Liu, Rynson W. H. Lau
Abstract:
While multimodal Large Language Models (MLLMs) excel at offline video understanding, an interesting question of how far they are from serving as a real‑time procedural coach remains unknown. Such a role typically requires an MLLM to continuously monitor the execution, detect mistakes, and provide corrective guidance in a closed‑loop interaction. In this paper, we construct GuideMe, the first multi‑domain benchmark for streaming video that supports training and evaluation of MLLMs for closed‑loop interactive task guidance. It comprises 2,458 videos spanning 223.7 hours across diverse domains (\eg, cooking, object manipulation, daily‑life guidance, and fitness), with 47,775 interaction samples covering next‑step instructions, completion feedback, error detection, and corrective guidance. To evaluate existing models on GuideMe, we design a three‑component assessment framework to measure the capabilities of representative MLLMs, which consists of temporal‑semantic bipartite matching for sequence‑level alignment, behavioral classification for intervention timing, and LLM‑as‑a‑Judge for content quality. Extensive experiments highlight a critical performance asymmetry: despite excelling at providing instructions, existing MLLMs consistently fail to identify execution errors and respond with corrective feedback. Code and data are released at https://fawnliu.github.io/project/guideme.
Authors:Wenzheng Zeng, Siyi Jiao, Chen Gao, Hwee Tou Ng, Mike Zheng Shou
Abstract:
Dense video captioning aims to generate temporally grounded descriptions of video events, benefiting both event‑level video understanding and generation. In this domain, autoregressive video large language models have emerged as a prevalent paradigm due to their strong generative and cross‑modal modeling capacity. However, generating dense captions under the token‑by‑token paradigm severely limits inference efficiency and hinders scalability as video length and event density increase. In this work, we propose a parallelized autoregressive framework that not only improves generation efficiency but also enhances temporally grounded captioning performance. Our key insight is to exploit the weak local dependencies across temporally distinct events to restructure the causal dependency graph, thereby enabling lossless parallel generation. Specifically, tokens with weak cross‑event dependencies can be decoded in parallel, while tightly coupled tokens within each event retain sequential decoding to preserve local semantic coherence. To realize this insight, we introduce two key components for lossless parallel decoding: (1) a latent global planning mechanism that automatically learns the event‑level structure and produces compact tokens encoding global inter‑event causality while adaptively aggregating event‑level audio‑visual semantics, guiding subsequent dependency restructuring and parallel decoding; and (2) an event‑factorized parallel decoding mechanism that effectively balances local focus with global inter‑event awareness. Experiments on various benchmarks demonstrate the clear advantage of our approach in both efficiency and performance in omni‑modal event grounding and captioning. Project website: https://github.com/showlab/PadCaptioner.
Authors:Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali
Abstract:
We introduce VSeek, an agentic framework that transforms long‑video question answering (LVQA) from a passive, single‑pass perception task into a multi‑turn retrieval process. VSeek utilizes a natural language‑driven search to identify relevant context within long videos and is post‑trained with reinforcement learning (RL) to jointly formulate targeted search queries and reason over retrieved clips for LVQA. While RL post‑training has revolutionized reasoning in symbolic domains such as mathematics and code, its application to long‑video understanding remains hindered by a lack of verified rewards. To ensure that the retrieved context is relevant, we propose a novel neuro‑symbolic approach that bridges open‑ended natural language with discrete visual verification. Specifically, complex user queries are compiled into formal temporal logic specifications for systematically decomposing natural language questions into a definitive checklist of required atomic visual primitives, such as key objects and activities, along with their temporal ordering. These systematically derived grounding events provide the critical feedback signal for RL post‑training, enabling dense, verifiable rewards based on the successful retrieval of these specific visual elements rather than relying entirely on outcome‑only answer accuracy. By explicitly optimizing for this verifiable evidence‑seeking behavior, VSeek improves Pass@1 scores by up to 8% and Pass@4 scores by 15% on long‑video understanding benchmarks compared to base models. We open‑source our code at https://utaustin‑swarmlab.github.io/VSeek.
Authors:Chaoqun Wang, Yuehuan Wei, Haoxiang Cao, Shaobo Min
Abstract:
Single‑image reflection removal (SIRR) aims to recover the clean transmission layer from a reflection‑contaminated image. Although recent methods achieve promising results with large diffusion models, they rely on image‑agnostic adaptation strategies, e.g., fine‑tuning or ControlNet, that enforce uniform suppression regardless of reflection severity. As a result, heavy reflections often leave residuals, while weak ones suffer from detail loss. To this end, we propose ReLo‑IRR, a reflection‑guided LoRA framework built upon the rectified flow model. First, a lightweight estimator is designed to predict the reflection strength descriptor, providing an explicit prior of reflection dominance for each image and enabling image‑dependent LoRA modulation. Second, we introduce a time‑conditioned mechanism that fuses this reflection descriptor with timestep embeddings, enabling LoRA modulation to evolve consistently with the coarse‑to‑fine denoising process. By jointly modeling reflection strength and denoising dynamics, our ReLo‑IRR achieves robust suppression of diverse reflection conditions. Extensive experiments on challenging benchmarks validate the effectiveness of ReLo‑IRR, demonstrating superior dereflection performance and robust generalization. The code is released at https://github.com/KONGBAI‑8080/ReLo‑IRR.
Authors:Long Xu, Binghong Wu, Tinghao Yu, Hao Feng, Zhenyu Huang, Haoqing Jiang, Yunhao Wang, Shuo Huang, Feng Zhang
Abstract:
Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine‑readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high‑resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision‑Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large‑scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real‑world documents via a model‑assisted, human‑refined annotation pipeline. We evaluate state‑of‑the‑art models using MORE, establishing new performance baselines for long‑tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.
Authors:Wen Dong, Zhao Wang, Shuangqing Zhang, Kai Sun, Ben Li, Guo-Sen Xie, Caifeng Shan, Fang Zhao
Abstract:
Multimodal Large Language Models (MLLMs) excel in diverse vision tasks, but full‑parameter retraining is computationally expensive as real‑world knowledge evolves. Existing continual learning methods often suffer from semantic entanglement in parameter spaces across tasks, impeding the continuous deployment of models. This challenge is especially pronounced in Anomaly Detection (AD), which exhibits triple heterogeneity across modalities, domains, and defect scale variability, significantly complicating multi‑task knowledge transfer. In this paper, we propose CL‑Anomaly, a parameter‑efficient fine‑tuning framework based on an isolation‑sharing collaboration to enable continual learning for anomaly detection with MLLMs. We introduce the task‑private expert PrivLoRA, which physically isolates task‑specific subspaces in the parameter space to prevent semantic entanglement of anomaly knowledge in diverse scenarios. The Layer‑Adaptive Shared Experts maintain cross‑task representations within a unified feature space, enabling knowledge sharing between previous and new tasks. Furthermore, we propose a Layer‑Adaptive Knowledge Transfer strategy that automatically selects and dynamically updates the layer‑wise key shared experts of each task via a momentum‑based mechanism, promoting effective knowledge transfer across related anomaly detection tasks. Extensive experiments across three continual learning scenarios for anomaly detection, including class‑incremental, cross‑domain, and cross‑modal, demonstrate that CL‑Anomaly outperforms state‑of‑the‑art methods. Code is available at https://github.com/WenDongyp/CL‑Anomaly.
Authors:Zhenkun Gao, Yicheng Bao, Jinlong Peng, Xueheng Li, Theo Huang, Bangwei Liu, Kunquan Li, Zhenye Gan, Tao Hu, Chengjun Xie, Mingqian Yang, Xuanhua He, Zhizhong Zhang, Xin Tan, Chengjie Wang, Yuan Xie
Abstract:
Video understanding is moving beyond closed‑context perception toward open‑world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text‑centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed‑loop agentic framework that empowers Vision‑Language Models with multi‑tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual clues, retrieve relevant evidence, and synthesize answers. To optimize knowledge‑intensive reasoning trajectories, we propose Bi‑branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool‑invocation optimization from answer‑accuracy optimization. This design provides stable learning signals for both evidence‑grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch‑QA, the first benchmark designed to evaluate open‑world video information grounding and multimodal search‑based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open‑source agentic baselines across various search‑oriented and multimodal understanding benchmarks.
Authors:Syed Ariff Syed Hesham, Yun Liu, Guolei Sun, Jing Yang, Henghui Ding, Xue Geng, Xudong Jiang
Abstract:
Video reasoning segmentation demands pixel‑accurate object tracking across hundreds of frames under complex natural language queries, producing dense spatiotemporal tokens whose quadratic self‑attention cost makes long‑video processing prohibitive. Existing methods address this through token compression, yet typically operate on encoder features lacking temporal context, constraining selection before content redundancy can be reliably assessed. Informed compression requires contextual awareness, but acquiring that awareness at full resolution incurs the same quadratic cost compression aims to reduce. State‑space models resolve this constraint, as their linear recurrence selectively conditions each token on temporal context at \mathcalO(T) cost, producing representations where content redundancy becomes assessable. Building on this, Selective SpatioTemporal Aggregation and Compression (STAC) enriches features via decoupled bidirectional spatial and causal temporal scanning, leveraging recurrence‑derived redundancy for hierarchical compression with adaptive thresholds optimised with segmentation objective. STAC achieves 85% token reduction and 1.8× speedup while surpassing compression‑free baselines on reasoning segmentation benchmarks in a zero‑shot streaming‑compatible setting. Code is available \hrefhttps://github.com/MCG‑NKU/nku‑videohere.
Authors:Jingjing Hu, Guo Dan, Haofan Cheng, Ying Zeng, Zhan Si, Jinxing Zhou, Meng Wang
Abstract:
Despite the high accuracy of EEG‑based emotion recognition, existing models remain opaque "black boxes", lacking semantic grounding between abstract neural features and human‑interpretable states. In this paper, we reframe EEG explainability as a cross‑modal generation task, shifting the paradigm from feature attribution to behavioral visualization. We introduce Facial Emoji Proxy Modeling, a novel framework that translates high‑dimensional EEG signals into identity‑anonymized facial emojis. Guided by the neuroscientific inspiration of neural‑facial association, this approach grounds neural representations in the manifold of observable facial dynamics. Technically, our framework integrates FMENet, a specialized backbone modeling expression‑relevant spatial synergies, and the Facial Emoji Learning Branch (FELB), which treats emoji reconstruction as a structured semantic regularizer. Extensive experiments on EAV and MMER benchmarks demonstrate that our method achieves state‑of‑the‑art accuracy among EEG‑only models. Crucially, it generates semantically faithful facial animations that provide a transparent, privacy‑preserving window into the brain's emotional evolution, effectively allowing users to "see the emotion" directly from neural signals. Code is available at https://github.com/xian‑sh/SeeEmotion
Authors:Hulingxiao He, Zhi Tan, Yuxin Peng
Abstract:
Taxonomies provide key information about the semantic relationships between concepts and the inherent organization of vision and language. Despite their impressive capabilities, large multimodal models (LMMs) often lack taxonomic knowledge, leading to low hierarchical visual recognition (HVR) consistency. These models typically only rely on language modeling objectives during fine‑tuning and lack explicit taxonomy‑aware regularization. To address this, we propose Hierarchical Representation Regularization (HiR^2), a simple plug‑and‑play regularizer that improves hierarchical consistency in LMMs. Specifically, we introduce a semantic‑aware visual tree construction framework that extracts coarse‑to‑fine visual features from intermediate LLM layers guided by textual cues. The regularizer combines two complementary objectives: a taxonomic entailment loss that enforces hierarchy via hyperbolic entailment cones in the Lorentz model, and a discriminative dispersive loss that promotes angular separation of semantically similar embeddings on the unit sphere without disturbing the radial hierarchical structure. Extensive experiments demonstrate that HiR^2 effectively captures taxonomic structures across diverse LMMs and fine‑tuning methods. Code is available at https://github.com/PKU‑ICST‑MIPL/HiR2_ICML2026.
Authors:Kun-Yu Lin, Chengke Bu, Zhenguo Li, Kai Han
Abstract:
This work introduces holo‑captioning, a novel task that strives to seek the text equivalent of 3D scenes. As the initial step, we formulate holo‑captioning as generating a structured textual description that comprehensively depicts all entities within a 3D scene ‑‑ including their semantic tags, spatial locations, attributes, and inter‑entity relations. To tackle this challenging task, we first develop an effective captioning engine to produce detailed descriptions of individual entity instances and instance pairs, and contribute a large‑scale benchmark comprising over 15K scenes for training and evaluation. Building upon this foundation, we propose HoloScribe, a novel model that features an instance‑aware decoupled pipeline for generating structured holo‑captions, and further incorporates anchor‑aware instance linking to identify relational instance pairs. Additionally, we propose a comprehensive evaluation metric named HoloScore, and provide a human‑curated test set to ensure reliable model assessment. Experimental results demonstrate that HoloScribe significantly outperforms state‑of‑the‑art 3D dense captioners and 3D LLM generalists, underscoring the effectiveness of our approach. Project page: https://visual‑ai.github.io/holocap/
Authors:Jongyeop Hyun, Hyounghun Kim
Abstract:
Deploying AI‑generated video detectors in real‑world services demands an ultra‑low false positive rate (FPR) on real videos to avoid falsely rejecting authentic content, a regime where standard metrics such as AUROC fail to reflect actual operating behavior. We introduce Spatial Patch‑Level Incoherence and Temporal Roughness (SPLIT), a training‑free detector that operates on patch tokens from a frozen vision encoder to detect both fully generated and partially edited videos. SPLIT computes two complementary signals: Two‑step Temporal Roughness (TTR), capturing non‑smooth patch trajectories via one‑step and two‑step feature variation contrast, and Local Spatial Motion Incoherence (LSMI), measuring spatially inconsistent temporal changes through gradients of a feature‑space motion field. The two are fused multiplicatively with gamma correction to sharpen real‑fake separation at strict thresholds. We further propose a service‑aligned evaluation protocol based on Fake Recall at fixed FPR with real‑only threshold calibration and cross‑real threshold transfer. Across three benchmarks (FakeParts, GenVideo, and ViF‑Bench), SPLIT achieves the highest Fake Recall at FPR = 0.1%, substantially outperforming supervised and training‑free baselines while remaining robust to post‑processing with negligible overhead. The code is publicly available at https://github.com/mldljyh/SPLIT .
Authors:Zhuoqun Li, Boxi Cao, Jiawei Chen, Hanshu Zhou, Ruoxi Xu, Guiping Jiang, Ruotong Pan, Tingting Gao, Han Li, Xiangyu Wu, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun
Abstract:
Long‑horizon behavior prediction aims to infer a user's next action based on a lengthy historical sequence, playing a crucial role in artificial intelligence field. The rise of large language models (LLMs) offers a promising direction for sequential behavior prediction, yet LLMs struggle with latent behavioral pattern induction and model‑intrinsic cognitive biases when tackling long‑horizon behavior prediction. Prior memory management methods follow a context‑compression paradigm that attempts to address this task by alleviating the historical sequence burden, yet fail to resolve the core challenges. In this paper, we advocate a paradigm shift that reframes the lengthy historical sequence from a burden into a valuable resource to be exploited, and accordingly propose PraMem, which conducts beforehand practice over the lengthy historical sequence to build an experiential memory, thereby serving as the assisted input for accurate long‑horizon behavior prediction. Extensive experiments across diverse tasks demonstrate that PraMem achieves superior performance than prior methods, and more in‑depth analyses provide valuable insights into the mechanism and evolution of the experiential memory. Code: https://github.com/icip‑cas/PraMem.
Authors:Weiying Chen, Junlong Shen, Zhanyuan Guo, Xiaoou Zhou
Abstract:
As LLMs are increasingly deployed as autonomous adjudicators in semi‑open textual game environments, robust rule adherence becomes critical when user intent conflicts with system rules. However, these models are trained to be helpful and compliant, leaving them vulnerable to a class of attacks we term Rhetorical Injection, where adversarial users exploit narrative framing techniques such as pseudo‑logical reasoning and authoritative coercion to bypass adjudication logic. We present CoC‑Seduce, a multi‑agent adversarial benchmark built on Tabletop Role‑Playing Game (TRPG) mechanics, an ideal instantiation of semi‑open environments where rules are explicit for adjudication, yet interaction remains entirely in natural language. Three frontier models, i.e., GPT‑5.4, Claude Sonnet 4.6, Gemini 3.5 Flash, serve as adversarial generators producing 5,376 samples across 4 world settings and 16 skill categories. We then benchmark 20 target adjudicators against this corpus. Evaluation across 20 models reveals that neither model scale nor explicit reasoning mechanisms reliably confer adjudication robustness, with \textscPseudo‑Logic emerging as the dominant attack vector and cross‑cultural settings exposing systematic knowledge gaps across all evaluated families. Project page: https://github.com/answerrtx/CoC‑Seduce
Authors:Yusheng Zheng, Zhengjie Ji, Weichen Tao, Xiangyu Gao, Jianchang Su, Wei Zhang, Andi Quinn, Dan Williams
Abstract:
eBPF lets developers run custom programs inside the Linux kernel, where a verifier proves each program safe. However, when the verifier rejects a program, the unclear error makes repair challenging: the error reports where verification stopped, not where the program lost the proof the verifier required. To quantify this gap, we conduct an empirical study of 235 reproduced rejections, showing that 47% of rejections return only EINVAL, one error string maps to as many as nine distinct root causes, and 10 of the 12 root causes are eBPF‑specific. Repair thus requires both domain knowledge and locating where the proof was lost, yet existing tools only help developers read the error. We present bpfix, which reconstructs where the required proof was established and where it was lost from the verifier log, and prints a Rust‑like diagnostic. To evaluate bpfix and the ability of LLMs to help repair, we construct a benchmark of 75 LLM repair tasks. Current models achieve 0‑37% one‑shot success with the raw log, and replacing the log with the bpfix localization improves repair by 11‑21pp, suggesting that locating where the proof was lost is key to guiding repair. bpfix is available at https://github.com/eunomia‑bpf/bpfix
Authors:Christophe Clienti
Abstract:
Network‑on‑Chip (NoC) architectures have become the standard interconnect fabric for many‑core systems, yet most proposals face a fundamental trade‑off between latency, area, and congestion management. This paper presents HyNoC (Hybrid Network‑on‑Chip), an open‑source NoC architecture that combines circuit‑switch path establishment with wormhole data transfer, targeting distributed computing systems built around VLIW processor cores on FPGA. HyNoC employs source routing, where the complete path through the network is encoded in the packet header by the sender or statically at compile time, enabling both deterministic low‑latency transfers and software‑level hotspot avoidance without the area overhead of virtual channels. The router features a parallel round‑robin arbiter (PRRA) with fixed grant latency, per‑port independent clock domains, and support for both unicast and multicast routing. We discuss the design rationale, the hybrid switching model, and positioning relative to prior NoC art, and argue that a richer topology combined with compiler‑assisted static routing is a competitive alternative to virtual channels for FPGA‑based distributed VLIW systems. Verilator co‑simulation measures the deterministic per‑hop latency and benchmarks the design at LLaMA 3 8B FFN up‑projection scale: a four‑master quadrant configuration partitions the mesh into traffic‑isolated quadrants with zero cross‑quadrant link traffic, yielding a 5x throughput improvement over a single master. The complete RTL implementation is available as open‑source hardware under the CERN‑OHL‑P v2 license.
Authors:Varshith Roy Kotla
Abstract:
Predicting thermal volatility in high‑performance EV powertrains is difficult as internal temperatures are rarely observable outside the lab, and models calibrated on lab drive cycles fail when deployed against real‑world loads. We study this lab‑to‑track transfer problem using conformal prediction, offering distribution‑free uncertainty bounds. We implement Ensemble Batch Prediction Intervals (EnbPI; Xu & Xie, 2021), a leave‑one‑out bootstrap‑ensemble conformal method for autocorrelated time series, and calibrate it on real CALCE lithium‑ion cycler data (A123 SP20 cells, FUDS profile). We evaluate it under a genuine, measured covariate shift: a second real CALCE test condition (US06 Highway Driving Schedule at 45°C). The unweighted EnbPI bound, achieving its nominal 95% coverage in‑distribution (measured: 95.00%), degrades to 70.13% empirical coverage under this real shift. We introduce a weighted EnbPI procedure combining EnbPI's ensemble residuals with density‑ratio weighting (Tibshirani et al., 2019), estimating the density ratio via a probabilistic domain classifier. This recovers coverage to 72.42%, a modest, honestly‑reported improvement, not a complete fix. We additionally apply the calibrated model to real 2023 Formula 1 telemetry (Monza and Silverstone, driver VER) as an unsupervised out‑of‑distribution diagnostic. Because no internal thermal channel exists in public trackside telemetry, we report only unsupervised flag rates (65.6% at Monza, 58.0% at Silverstone, well above the 5% in‑distribution base rate) and note inconsistent associations between flags and braking/DRS zones. We conclude that conformal domain adaptation is a promising but only partially solved tool for this problem, detailing exactly where it falls short.
Authors:Waseem Mousa, Alaa Maalouf
Abstract:
3D Gaussian Splatting (3DGS) enables high‑quality real‑time novel‑view synthesis, but practical scenes often contain millions of Gaussians, making compression essential for deployment on limited hardware. Existing reduction methods are effective but mostly heuristic: they provide no multiplicative approximation guarantee for the rendered objective, and thus rely heavily on costly post‑pruning finetuning to recover quality. We ask a basic question: can a 3DGS scene be provably replaced by a much smaller weighted subset (coreset) while preserving the objective of interest? We first show that, in the unrestricted setting, no non‑trivial multiplicative 3DGS coreset exists. We then show that multiplicative guarantees are not impossible, but resolution‑dependent. For a prescribed rendering resolution, such as representative views or grids of views/rays, we provide the first weighted coreset construction theorem for 3DGS. The construction samples Gaussians by sensitivity: provable importance scores measuring each Gaussian's role in the full‑scene objective. Finally, under explicit validity and log‑transmittance stability assumptions, we turn this objective guarantee into a rendering guarantee. Empirically, our method is strongest where deployment needs it most: aggressive compression with no or minimal recovery compute. In prune‑only and very short finetuning regimes, it achieves state‑of‑the‑art performance, showing that principled importance estimation can be both theoretically meaningful and practically useful. Open‑source code is available at https://github.com/waseem‑m/3dgs_provable_coresets.
Authors:OFM Riaz Rahman Aranya, Peyman Najafirad, Kevin Desai
Abstract:
Radiologists routinely compare current and prior chest X‑rays to track disease progression, producing follow‑up reports that describe multiple findings, each localised to an anatomical region and annotated with a temporal change status. Existing automated methods either generate reports from a single image without modelling temporal context, or incorporate temporal information but do not ground their outputs spatially. The few approaches that combine temporal reasoning with spatial grounding are restricted to single‑finding descriptions, leaving multi‑finding reports with mixed change directions unaddressed. We present GRCD, a framework for grounded report generation from chest X‑ray pairs in the multi‑finding setting. We first construct a rigorously cleaned dataset of temporal chest X‑ray pairs by identifying and correcting two systematic labelling errors in the source annotations. We then introduce a Region‑Guided Change Token module that encodes per‑region temporal change across anatomical structures and injects this signal into a language model through a dual‑pathway strategy combining prepended spatial tokens with gated cross‑attention. On a multi‑finding test set, GRCD outperforms existing baselines on text generation and clinical accuracy metrics, with gains in change detection. Ablation studies confirm that the dual‑pathway design outperforms either integration strategy in isolation on text and clinical metrics, and that region‑level change encoding is necessary for multi‑finding generation. Code is available at https://github.com/UTSA‑VIRLab/GRCD
Authors:Stanislav Panev, Minhyek Jeon, Vaishnavi Khindkar, Ahish Deshpande, Celso M de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre
Abstract:
Recent advances in large‑scale image generative models enable photorealistic scene synthesis with controllable attributes. Beyond data augmentation, their potential as diagnostic tools for trained vision systems remains unexplored in the aerial and remote sensing domains. We introduce a synthetic diagnostic framework for aerial‑view vehicle detection that combines text‑guided generation, attribute‑controlled editing, and automated attribute verification to construct a controllable synthetic testbed. This enables fine‑grained evaluation of pretrained detectors under diverse scene types and environmental conditions that are difficult to isolate in real datasets. Across three detection architectures and three real aerial datasets, synthetic scene‑wise performance trends closely match real‑world weaknesses. Guided by these diagnostics, targeted supplementation with small real datasets from the identified weak categories yields improvements of up to 13% AP50 while requiring substantially fewer additional samples than non‑targeted augmentation. Our results show that controlled synthetic probing can predict real‑domain performance gaps and guide efficient data collection. The proposed diagnostic framework is modular and can incorporate alternative generative or vision‑language models as capabilities evolve. Our code and datasets are available here: https://humansensinglab.github.io/AVODDiag/
Authors:Zhibing Li, Amogh Gupta, Behnoosh Parsa, Dan Casas
Abstract:
Novel View Synthesis (NVS) enables the generation of unseen views of a scene from a single or multiple images, allowing users to freely explore an object from any viewpoint. Despite the recent impressive qualitative improvements of generative models for this task, existing methods struggle to provide global and intuitive control of target viewpoints because they either use input‑relative camera poses or are limited to generating sparse global views. This lack of global pose control severely limits the number of downstream tasks potentially enabled by NVS. To address this limitation, we propose a novel approach for precise camera control in a customizable Normalized Object Coordinate Space (NOCS), requiring single or few unposed images. Our method operates solely on the absolute camera pose of the target view in NOCS, eliminating the need for a relative world frame or camera poses of the input images. Unlike previous methods that treat NVS as a standalone generation task, we formulate it as an image editing problem and build upon state‑of‑the‑art editing models to leverage their superior generalization capability. Camera information is injected as dedicated camera tokens via an in‑context multi‑modal conditioning strategy. To alleviate the inherent ambiguity of NOCS, we incorporate text descriptions that explicitly define the object's canonical coordinate frame, which also enhances generalization to unseen object categories. Furthermore, we curate a high‑quality dataset with consistently aligned orientations and corresponding NOCS text definitions. Extensive experiments demonstrate that our method robustly generates novel views with accurate and consistent orientations from arbitrary unposed images across diverse categories, achieving state‑of‑the‑art image quality and fidelity.
Authors:Khush Attarde, Yusuf Ali, Megha Thukral, Divye Bhutani, Thomas Ploetz, Zsolt Kira
Abstract:
MLLMs have shown strong zero‑shot capabilities across diverse inputs such as across images, video, audio, and text. A crucial, yet underexplored, application of these models lies in understanding and modeling animal‑centric scenarios. As animals are integral to millions of households, benchmarking next‑generation AI models on pet‑focused tasks, ranging from recognizing distress signals to enabling responsive robotic companions, is essential for building AI systems that can work alongside us. We introduce K9‑Bench, a novel benchmark focused on real‑world domestic dog videos, specifically targeting canine action and interaction understanding via approximately 5000 question‑answer pairs across 907 videos spanning 5 distinct task categories that test long‑form, canine‑centric multimodal reasoning in MLLMs. To create this dataset, we propose a scalable, VLM/LLM‑powered data generation pipeline that automatically mines canine‑centric videos from the web and curates QA pairs requiring fine‑grained, multi‑hop reasoning over canine actions and temporally extended interaction sequences. We implement bias mitigation strategies designed to eliminate biases introduced by VLMs during dataset curation. Through extensive experimentation, we find that frontier MLLMs exhibit limited zero‑shot performance on canine‑centric tasks: although state‑of‑the‑art closed‑source models outperform open‑source counterparts, they still struggle with compositional reasoning over subtle posture and interaction cues spread over long horizons. We observe that generic chain‑of‑thought prompting provides only modest performance for such long‑horizon reasoning. Beyond a novel dataset for canine activity analysis, K9‑Bench provides a general‑purpose dataset construction pipeline that can be adapted to other low‑data domains for quantitative analysis. Our project website is available at: https://ogmenrobotics.github.io/K9Bench.
Authors:Haoran Wang, Mohit Mendiratta, Christian Theobalt, Adam Kortylewski
Abstract:
Precise control of 3D facial expressions from text is crucial for virtual avatars, animation, and human‑computer interaction, yet existing text‑to‑3D methods jointly generate identity, expression, and texture, making fine‑grained expression control difficult. We instead formulate text‑driven expression synthesis as a regression problem in the disentangled parameter space of a 3D Morphable Model (3DMM). This setting, however, requires paired data linking detailed language to precise expression parameters, which are missing from existing resources. To fill this gap, we introduce Txt2Emote, a benchmark of diverse 3D facial expressions with fine‑grained textual annotations obtained from GPT‑4o and a high‑fidelity face tracker, providing both explicit descriptions detailing facial features and implicit descriptions referencing the situational context behind the expression. Leveraging this dataset, we present EmoteGPT, a text‑to‑3D expression framework based on a Multimodal Large Language Model (MLLM) with a dedicated <Expr> token to semantically ground expression representations, which are then decoded into 3DMM parameters. We further improve EmoteGPT by augmenting training with large‑scale image‑to‑3DMM data, enabling it to surpass state‑of‑the‑art text‑to‑3D face synthesis methods on emotion recognition metrics and in perceived expressiveness. Integrated into avatar pipelines, our method enables photorealistic and stylized 3D avatars, as well as expressive 3D‑consistent 2D face synthesis from textual input.
Authors:GigaWorld Team, Angyuan Ma, Boyuan Wang, Bohan Li, Chaojun Ni, Guo Li, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jingyu Liu, Jiwen Lu, Qiuping Deng, Tingdong Yu, Xuancheng Xu, Xinyu Zhou, Xiuwei Xu, Xinze Chen, Xiaofeng Wang, Xiaoyu Tian, Yang Wang, Yifan Chang, Yukun Zhou, Yun Ye, Zhenyu Wu, Zhanqian Wu, Zheng Zhu
Abstract:
Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real‑world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real‑robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large‑scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long‑horizon, action‑faithful rollout consistency rather than short‑term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot‑specific controllability; and architectural choices including action encoding, memory design, and evaluator‑focused post‑training strongly determine alignment with real‑world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in GigaWorld‑1, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.
Authors:Yasser Alemán-Gómez, Nino Hervé, Patric Hagmann
Abstract:
Neuroimaging research requires manipulating heterogeneous data structures, including raw MRI volumes, volumetric parcellations, cortical surface meshes, tractograms, and connectivity matrices, across tools with incompatible interfaces and file formats, forcing researchers to repeatedly re‑implement routine but technically demanding operations. We present CLABTOOLKIT, an open‑source Python package that consolidates these operations into a single, coherent framework by representing volumetric, surface, and streamline data as interoperable Python objects. Five core data structures (Parcellation, Surface, AnnotParcellation, Tractogram, and Connectome) encapsulate common neuroanatomical entities and provide consistent methods for loading, processing, and exporting data across standard neuroimaging formats (e.g., NIfTI, GIFTI, FreeSurfer annotations, TCK/TRK), including connectome generation from a parcellation and scalar‑map projection onto tractogram streamlines. Complementary modules support BIDS dataset management, FreeSurfer integration, diffusion MRI processing, morphometric analysis, graph‑theoretical network analysis, and GPU‑accelerated multi‑panel visualization via PyVista. The toolkit comprises 19 modules organised into six layers, exposing 13 object‑oriented classes with 234 methods and 207 standalone functions, and a JSON‑based configuration system enables workflow customization without code changes. Unlike existing neuroimaging libraries, which typically address these tasks separately, CLABTOOLKIT combines color and lookup‑table management, parcellation manipulation, multi‑surface visualization, and tractography utilities within a single framework. CLABTOOLKIT is compatible with Python 3.9‑3.12 and released under the Apache 2.0 license. Source code, documentation, and example workflows are available at https://github.com/connectomicslab/clabtoolkit.
Authors:Matthias Reumann, Yannick Stade, Robert Wille, Lukas Burgholzer
Abstract:
The Multi‑Level Intermediate Representation (MLIR) framework has become a cornerstone for building extensible, domain‑specific compilers, with the quantum computing community already leveraging it to model quantum programs and implement basic optimizations. However, computationally intensive tasks in the quantum compilation pipeline, such as quantum circuit mapping, remain underexplored within the MLIR ecosystem. This paper proposes an MLIR‑native blueprint for these non‑local, quantum‑specific optimization routines by reimplementing a well‑established, state‑of‑the‑art mapping A search algorithm for qubit routing and SWAP insertion. Our evaluation demonstrates that this approach not only integrates seamlessly into an MLIR‑based quantum compiler collection but also surpasses previous non‑MLIR solutions in both solution quality and runtime. The implementation is open‑source and publicly available at https://github.com/munich‑quantum‑toolkit/core.
Authors:Wanyi Chen, Daoyuan Chen, Fang Kong
Abstract:
Structured‑output benchmarks reward both task decisions and interface compliance, so prompt‑induced function‑calling gains require attribution before they can be interpreted as transferable skill. We introduce a four‑layer gain‑attribution protocol for prompt‑prepended skill injection, combining canonicalized rescoring, format‑only controls, repaired/balanced induction, and portability checks. Applied to the Berkeley Function Calling Leaderboard (BFCL) and scoped with API‑Bank, MATH‑500, and MultiHop‑RAG, the protocol shows that several apparent gains are better attributed to interface alignment than to procedural transfer: format‑only prompts match or exceed full skills in key BFCL cells, repaired/balanced induction removes the largest sub‑frontier gains, and API‑Bank target‑native gains are matched within 0.5 percentage points (pp) by length‑matched generic procedural prompts. These findings treat format compliance as a useful engineering capability while clarifying what a structured‑output score certifies. We release BFCL‑CANONICAL and recommend canonicalized metrics, balanced induction, and format‑only baselines for function‑calling skill‑gain attribution. Code and data are available at https://github.com/couragec/skill‑injection‑attribution.
Authors:Qixiang Yin, Huanjin Yao, Yuchen Cai, Jianghao Chen, Ziyi Wang, Min Yang, Fei Su, Zhicheng Zhao
Abstract:
On‑policy distillation (OPD) has recently emerged as an effective post‑training paradigm by providing supervision on student‑generated trajectories. However, existing OPD methods for multimodal reasoning usually rely on a static teacher routing, assigning each sample to a single teacher based on modality or task type. This ignores that visual grounding and abstract reasoning may dominate different decoding steps, making a single teacher insufficient for the full trajectory. To this end, H‑OPD is proposed as a confidence‑aware heterogeneous multi‑teacher OPD framework for multimodal reasoning. By verifying the complementarity of heterogeneous teachers in the same reasoning process, H‑OPD replaces task or sample level teacher routing with token‑level teacher arbitration along the shared student trajectory. H‑OPD employs vision‑to‑language description transfer to enable text‑only teachers to access key visual semantics, and uses a confidence‑aware arbitration mechanism to dynamically combine vision‑language teacher and text‑only teachers at each token. Extensive evaluations over 11 widely‑used reasoning benchmarks showcase the superior performance of our method.
Authors:Ying Chen, Jinyue Li, Kun Wang, Qiankun Li, Yang Liu
Abstract:
The Segment Anything Model with Concepts (SAM3) heralds a new paradigm for open‑vocabulary segmentation through natural language interaction, offering significant potential for medical image analysis. However, effectively adapting such a powerful vision‑language model to the diverse and nuanced domain of medical imaging remains a key challenge. Naive fine‑tuning is parameter‑inefficient, while standard Mixture‑of‑Experts (MoE) methods introduce prohibitive computational overhead, limiting their clinical applicability. To address this, we propose Dual‑Adaptive SAM3 (DA‑SAM3), a novel framework that achieves both high segmentation accuracy and extreme parameter efficiency via a dual‑adaptive specialization mechanism. Our first adaptation is task‑aware: a Dynamic Expert Router (DER) that sparsely activates the most relevant experts by jointly reasoning about the visual input and the textual concept prompt, mimicking a clinical consultation process. Our second adaptation is parameter‑aware: a Decomposed Parameterized Experts (DPE) design that represents each expert as a shared frozen base (inherited from the pretrained SAM3) and a lightweight trainable low‑rank delta, reducing MoE parameter overhead by over 80%. Extensive experiments on multiple public medical segmentation benchmarks demonstrate that Dual‑Adaptive SAM3 not only matches or exceeds the accuracy of fully fine‑tuned SAM3 and standard MoE baselines, but also achieves a notable 5% gain over current state‑of‑the‑art methods, with interpretable results validating its effectiveness. The code is available at: https://github.com/Reconsider80/DA‑SAM3.
Authors:Karim Mardhani
Abstract:
This paper presents a safety‑centered empirical evaluation of uncertainty‑aware last‑layer adaptation for referable diabetic retinopathy screening using RETFound, a self‑supervised vision‑transformer retinal foundation model used here as a frozen feature encoder, and the public APTOS 2019 and DDR diabetic retinopathy fundus image datasets. We compare a cached‑feature softmax head, post‑hoc temperature scaling, variational Bayesian last‑layer heads, a diagonal Laplace last‑layer approximation, and an SNGP‑style cached‑feature head. On APTOS, uncertainty‑aware operating points improved sensitivity and selective‑referral behavior. The strongest APTOS selective‑referral result deferred approximately 20 percent of cases and reduced accepted‑case false negatives to zero while preserving high accepted‑case specificity. However, threshold tuning also reduced false negatives at high false‑positive cost, so false‑negative reduction alone was not unique to Bayesian modeling. On DDR, native Bayesian heads qualitatively reproduced the APTOS direction but with weaker tradeoffs, while the APTOS‑trained SNGP checkpoint transferred poorly and failed to provide useful external selective‑referral behavior. These results highlight the value of safety‑centered evaluation beyond aggregate accuracy: uncertainty‑aware last‑layer heads can improve internal safety‑oriented operating points, but trustworthy retinal screening claims require explicit safety‑coverage evaluation and second‑dataset validation under shift.
Authors:Weize Quan, Zhengwei Wu, Kai Wang, Dong-Ming Yan
Abstract:
View‑based point cloud completion aims to recover a complete 3D shape from a partial point cloud, guided by a single‑view image. However, existing approaches often suffer from limited performance due to weak modality alignment and limited self‑geometry enhancement. To overcome these challenges, we propose a unified geometry‑aware framework that integrates efficient modality alignment and adaptive geometry enhancement, mainly to address cross‑modal geometric inconsistency of view‑guided point cloud completion. Specifically, we propose a geometry‑aware modality alignment by integrating a shared self‑attention Transformer and cross‑modality reconstruction supervision, which aims to bring features of the image and point cloud close to each other in a shared latent space describing the 3D object. To enhance the perception of global shape and local geometric details, we propose an adaptive geometry‑aware self‑attention module, which simultaneously considers local geometry‑aware attention computation and the spatially‑variant feature fusion. In addition, we apply a geometry‑perceptive anchor refinement module to reorganize the anchor points (representing a local region of the shape) under appropriate supervision, further boosting the completion performance of our method. Extensive experiments on both synthetic and real‑world datasets demonstrate that our method achieves superior performance over existing approaches. Our code will be available at https://github.com/weizequan/MAGE.
Authors:Chakshu Baweja
Abstract:
Lunar positioning, navigation, and timing (PNT) is moving from concept to hardware, ESA's Moonlight/LCNS, NovaMoon reference stations, LunaNet, and Coordinated Lunar Time, all reducing to one estimation core: fix the orbits and clocks of the lunar infrastructure and tie them to an Earth/inertial frame. We ask which measurements make a surface station's absolute position observable, and prove the answer. In a snapshot batch fit, the internal observables (station‑to‑satellite and inter‑satellite ranging plus clock‑sync) constrain only relative geometry and leave a six‑dimensional rigid‑body datum defect: three translations and three rotations of the cluster. The clocks are fully observable, so the defect is purely positional, and closing it needs a tie to the Earth frame. Two such ties exist and are not interchangeable. An indirect tie (Earth‑to‑satellite ranging through the constellation) reaches the station only when the satellite geometry is rich; a direct tie (an Earth‑baseline VLBI delay to the station beacon) fixes it regardless. This gives a conditional design law, not a single number: VLBI restores absolute observability when the constellation cannot supply it, and merely sharpens the bound when it can. For a sparse three‑satellite constellation the station lies in the null space of the Fisher information until VLBI is added, reaching a Cramer‑Rao bound of 20.1 m; for a rich six‑satellite constellation VLBI tightens the bound from 23.2 m to 9.7 m. A single‑epoch baseline informs at most two of three axes, so the datum closes at three non‑collinear Earth stations. The Gauss‑Newton estimator attains the bound (efficiency 1.02), with a 91x median station‑error improvement in the sparse regime. The FIM/CRLB engine is validated against NumPy and published closed forms; the lunar application stays modelled, every figure deterministic and reproducible.
Authors:Azim Akhtarshenas, Mario Rico Ibanez, Matteo Bernabe, David Lopez-Perez, Merouane Debbah
Abstract:
Unmanned aerial vehicle‑mounted base stations (UAV‑BSs) constitute a flexible and effective solution for global positioning system (GPS)‑free emergency and disaster scenarios, where the rapid deployment of communication infrastructure is critical for maximizing life‑saving operations. In this work, we extend a centralized learning framework to a multi‑UAV‑BS network architecture, in which a single centralized UAV‑BS ‑‑ as an intelligent agent ‑‑ coordinates the three‑dimensional positioning and navigation of multiple UAV‑BSs, while the remaining UAV‑BSs actively serve ground user equipments (UEs) with uncertain positions. We formulate a fairness‑aware sum‑throughput maximization problem for UAV‑BS coordination, which is inherently nonconvex due to the non‑linear and interference‑coupled throughput expressions. To address this challenge, we cast the problem as a Markov Decision Process (MDP) and solve it using a deep reinforcement learning (DRL) framework based on Proximal Policy Optimization (PPO). The central agent interacts with the environment and learns optimal joint positioning policies that guide the serving UAV‑BSs to provide efficient, adaptive, and resilient wireless coverage. The proposed approach exploits spatial configuration and radio signal sensing capabilities to dynamically adapt to heterogeneous UE mobility patterns. Extensive simulations are conducted to evaluate the performance of the proposed method. Numerical results demonstrate that PPO shows competitive performance during both training and evaluation phases. Furthermore, comparative analysis with state‑of‑the‑art RL algorithms, namely Deep Deterministic Policy Gradient (DDPG) and Deep QNetwork (DQN), shows that PPO consistently outperforms these methods in terms of convergence stability, mean reward, and network throughput.
Authors:Shamsher Khan
Abstract:
Kubernetes clusters generate rich operational events during pod lifecycle transitions, yet the platform's native event retention model systematically discards the most diagnostically valuable context through multiple evidence destruction mechanisms operating on deterministic schedules. We formalize these mechanisms as an evidence horizon taxonomy: five distinct boundaries after which specific categories of diagnostic context become permanently unrecoverable from the Kubernetes API. H1(LastTerminationState rotation, ~90s) destroys container failure forensics; H2 (scheduler event pruning, 1hr/1000‑event cluster limit) destroys placement rationale; H3 (ephemeral container exit, immediate) destroys debug session context; H4 (kubelet reconciliation gap) destroys in‑memory operational state; and H5 (scrape‑interval blind spot) renders sub‑interval pod lifetimes invisible to poll‑based observability tools. This paper extends the Operational Memory Architecture (OMA) to address the full evidence horizon taxonomy. Two new causal patterns are defined: P004 (Scheduler Decision Provenance) captures FailedScheduling predicate failures before kube‑apiserver TTL pruning and demonstrates the first cross‑horizon causal chain linking scheduler evidence to downstream OOMKill failures. Two new Go watchers (EventWatcher, EphemeralWatcher) and two new storage tables (scheduler_events, ephemeral_exits) extend the original architecture. Validated on Minikube (3‑node) and AKS 1.32.10. The original 30‑run statistical latency analysis (242 edges, intra‑cycle mean 0.702ms) and stress evaluation (2.86 events/sec at 20 pods, 8.8MB RAM) are carried forward and augmented with H2, H3, and H5 results.
Authors:Rajesh Kumar, Waqar Ali, Junaid Ahmed, Abdullah Aman Khan, Shaoning Zeng
Abstract:
Automated research agents increasingly generate code, retrieve literature, and draft scientific artifacts, but they often fail to verify whether generated experiments execute correctly or whether cited sources support generated claims. We present AutoResearch, an execution‑grounded multi‑agent framework for reliable research workflow automation. AutoResearch couples sandboxed Python/PyTorch execution, iterative code repair, citation verification, claim‑support auditing, decision control, and structured \LaTeX artifact generation. The system treats runtime errors, citation‑verification failures, and review‑agent feedback as practical filtering signals for generated research artifacts. In controlled evaluations on HumanEval, MBPP, a SciCode subset, citation‑validation tasks, claim‑support auditing, and small end‑to‑end workflow stress tests, AutoResearch improves execution success, citation validity, local claim support, and workflow completion relative to directly comparable baselines. Code‑oriented agents are reported separately as partial comparisons. AutoResearch is intended as a reliability‑oriented research assistant, not as a fully autonomous scientist or a standalone manuscript‑quality benchmark. Source Code: https://github.com/raja21068/AutoResearch
Authors:Qiaowei Miao, Kehan Li, Yawei Luo, Yi Yang
Abstract:
Generative diffusion models excel at synthesizing high‑quality images, videos, and 3D content under multimodal control. However, arbitrary user‑defined modality‑to‑4D (X‑to‑4D) generation remains challenging due to the high cost of constructing diverse datasets and the limited scalability of existing methods. This paper presents Align4D, a flexible framework that translates any‑modal input into coherent video‑3D pairs, using video to guide 4D motion and 3D data to shape 4D geometry. Align4D introduces three key techniques: (1) Object Distance Alignment, which searches Video‑Aligned and Multiview‑Aligned Object Distances (VAOD/MAOD), respectively, to reconcile 4D renderings with video and the priors of multiview diffusion models; (2) Motion‑Geometry Joint Alignment, which constrains known and unknown views through synchronized video and 3D inputs, ensuring consistent 4D generation; and (3) Asynchronous Optimization, which decouples Gaussian attribute and deformation network training to enhance motion and geometry fidelity. We further propose the X4D dataset, which integrates prompt, image, video, and 3D data for benchmarking. Experiments on X4D and Consistent4D demonstrate that Align4D achieves state‑of‑the‑art quality and consistency in X‑to‑4D generation. Project page: https://miaoqiaowei.github.io/Align4D/.
Authors:Haofei Xu, Rundi Wu, Philipp Henzler, Nikolai Kalischek, Michael Oechsle, Fabian Manhardt, Marc Pollefeys, Andreas Geiger, Federico Tombari, Michael Niemeyer
Abstract:
State‑of‑the‑art single‑image 3D reconstruction methods often rely on complex hybrid architectures and loss functions, or compress geometry into latent spaces in order to leverage pre‑trained latent diffusion models. In this work, we show that such architectural overhead and intricate loss formulations are unnecessary. We introduce a minimalist pixel‑space Diffusion Transformer, built on a plain ViT, that operates directly on raw 3D point map patches and is conditioned on image tokens from a pre‑trained DINOv3. Unlike existing latent diffusion approaches, we train our diffusion backbone entirely from scratch, eliminating the need for point map tokenizers. Despite its simplicity, our approach surpasses complex latent‑based diffusion models while remaining significantly simpler than hybrid alternatives. Notably, it produces sharper geometric structure and is more robust in highly ambiguous regions, such as transparent objects.
Authors:Yanjun Zhao, Ruizhong Qiu, Tianxin Wei, Yuanchen Bei, Zhining Liu, Lingjie Chen, Ismini Lourentzou, Hanghang Tong, Jingrui He
Abstract:
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long‑Context Reasoning (RECONTEXT), a training‑free inference method for improving long‑context reasoning. RECONTEXT uses model‑internal relevance signals to construct a query‑conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue‑trace association, and replay as trace reactivation. Experiments on eight long‑context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3‑4B, Qwen3‑8B, and Llama3‑8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun‑Zhao/ReContext.
Authors:Shuai Tian, Yupeng Zheng, Yuhang Zheng, Songen Gu, Yujie Zang, Yuxing Qin, Weize Li, Haoran Li, Wenchao Ding, Dongbin Zhao
Abstract:
Contact‑rich manipulation requires policies to react to local deformation, pressure, slip, and friction, yet these cues are temporally sparse and often invisible in visual observations. Existing visual‑tactile policies usually feed tactile observations directly into action prediction, but rarely model tactile deformation dynamics during action generation. In this paper, we introduce VT‑WAM, a Visual‑Tactile World Action Model that jointly learns future visual prediction, tactile deformation prediction, and action prediction within a unified flow matching framework. In particular, VT‑WAM introduces (1) Asymmetric Mixture‑of‑Transformers (MoT) attention to bridge a first‑frame visual anchor with temporal tactile dynamics, and (2) contact‑gated Action‑Visual‑Tactile Attention Guidance (AVTAG) to encourage action queries to rely on tactile evidence during contact phases. Across six real‑world contact‑rich manipulation tasks, VT‑WAM achieves a 71.67% average success rate, outperforming Fast‑WAM by 26.67% and OmniVTLA by 35.84%. Ablations demonstrate that modeling tactile deformation dynamics and guiding contact‑phase tactile attention are both important for contact‑rich tasks. Project website: https://vt‑wam.github.io/.
Authors:Ling Xu, Chuyu Han, Borui Li, Hao Wu, Shiqi Jiang, Ting Cao, Chuanyou Li, Sheng Zhong, Shuai Wang
Abstract:
Embodied AI models now span vision‑language‑action (VLA) models and world‑action models (WAMs), but practical deployment remains fragmented across model‑specific Python stacks, backend assumptions, and robot‑side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request‑response serving and therefore do not satisfy the runtime contract of embodied deployment: multi‑rate execution inside closed‑loop control, latency‑first batch‑1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied.cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied.cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi‑rate execution, latency‑first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied.cpp on two VLA models, HY‑VLA and pi0.5, and on a preliminary WAM benchmark using a LingBot‑VA Transformer block. The VLA deployments achieve successful closed‑loop execution with 100.0% and 91.0% task success rates, respectively. The WAM benchmark reduces block memory from 312.2 MiB to 88.1 MiB. These results show that Embodied.cpp improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.
Authors:Bohan Liu, Wenqian Ye, Guangzhi Xiong, Zhenghao He, Sanchit Sinha, Aidong Zhang
Abstract:
Models trained via Contrastive Language‑Image Pretraining (CLIP) serve as the foundational vision encoders for most modern Large Vision Language Models (LVLMs). Despite their widespread adoption, CLIP models exhibit a critical yet underexplored failure mode: irrelevant text appearing within images confounds visual representations, biasing them toward lexical meaning rather than true visual semantics. This robustness issue, commonly described as a Typographic Attack (TA), exposes a vulnerability that poses a significant risk to safety‑critical applications such as autonomous driving. To achieve interpretable and effective robustness against TA, we propose a novel, training‑free mechanistic interpretability method. Our method provides sampling‑based interpretations of hidden state representations and quantitatively attributes semantic versus lexical focus to individual attention heads. Through probabilistic analysis and circuit mining, we isolate specific Vision Transformer (ViT) components that disproportionately encode lexical information, thereby identifying the mechanistic source of TA. We further show that simple interventions applied directly to the identified circuits, without any additional training, can substantially improve robustness against Typographic Attacks in object classification. These interventions, such as selective adjustment of attention weights, also outperform both supervised and training‑free defense methods. Our experiments demonstrate that applying the proposed intervention to the vision encoders of several state‑of‑the‑art LVLMs yields substantial gains in Visual Question Answering accuracy under Typographic Attack interference on RIO‑Bench. These results confirm both the efficacy and the generalizability of our mechanistic approach. Code is released at https://github.com/Liu‑524/SamplingTAR.
Authors:Yejun Zhang, Xinjue Wang, Zihan Wang, Esa Rahtu, Juho Kannala
Abstract:
Descriptor‑free visual localization eliminates high‑dimensional descriptor storage, preserves scene privacy, and simplifies map maintenance, yet its accuracy still lags far behind descriptor‑based pipelines. We identify this gap to insufficient geometric discriminability in geometry‑only matching. Without visual appearance, current methods underutilize local geometry cues, lack the global context among keypoints, and overfit to a single keypoint detector. We further observe that descriptor‑free matching naturally enables multi‑detector training, as heterogeneous keypoints can be optimized in a shared geometry‑only space without aligning descriptor spaces. Building on these insights, we propose GeoMix, a descriptor‑free 2D‑3D matching framework that strengthens geometric discriminability at three levels. Locally, directional and distance‑aware embeddings enrich neighborhood aggregation with fine‑grained spatial structure. Globally, learnable context nodes aggregate and redistribute scene‑wide information via cross‑attention to resolve ambiguities beyond local receptive fields. At the training level, Mix‑Training exploits this detector‑agnostic geometry space to learn representations across multiple keypoint detectors. Extensive experiments on MegaDepth, Cambridge Landmarks, 7Scenes, and Aachen Day‑Night show that GeoMix sets a new state of the art among descriptor‑free methods, reducing 75th‑percentile rotation error by 89% and translation error by up to 90% over the previous best, while generalizing zero‑shot to unseen detectors and narrowing the gap to descriptor‑based pipelines. Code is available at \hrefhttps://github.com/YejunZhang/Geomix\textthis links.
Authors:Ziyao Wang, Maonan Wang, Yucheng He, Xianping Ma, Ziyi Wang, Hongyang Zhang, Yirong Cheng, Man-on Pun
Abstract:
Cloud removal (CR) is essential for optical remote sensing, serving as a prerequisite for reliable downstream interpretation, such as semantic segmentation and change detection. However, existing CR approaches often prioritize visual realism while overlooking their impact on subsequent analytical tasks, leading to semantic drift and degraded downstream performance. To address this issue, we propose Geo‑Anchored Cloud Removal (GACR), a unified framework that jointly ensures faithful reconstruction and robust interpretability. At its core, GACR incorporates Observation‑Anchored Residual Flow (OAR‑Flow), which reformulates CR as a physically grounded residual inversion process. By anchoring the generative trajectory to the cloudy observation rather than pure noise, OAR‑Flow enables fast, stable, and faithful reconstruction. To further preserve semantic structures critical for downstream interpretation, GACR integrates Geo‑Contextual Prior Alignment (GCPA) to constrain the reconstruction within a semantic manifold induced by a Vision Foundation Model (VFM). Consequently, GACR strictly maintains the spatial‑semantic integrity of complex landscapes. Extensive experiments across six CR datasets and twelve downstream tasks demonstrate that GACR produces superior reconstruction quality while consistently improving downstream task accuracy. The code is available at https://github.com/wzy6055/GACR.
Authors:A. S. Anudeep, Vaanathi Sundaresan
Abstract:
For clinical deployment, it is essential that automated diagnostic systems remain reliable when confronted with previously unseen cases, yet deep models routinely misclassify out‑of‑distribution (OOD) inputs with high confidence, underscoring the need for more robust OOD detection methods. Although substantial effort has been devoted to improving model robustness, most of the existing literature assumes balanced datasets, evaluates OOD detection on coarse or non‑clinical OOD sources, or lacks comprehensive assessment across diverse OOD scenarios. To address the gaps, we propose a novel methodology trained on diverse and imbalanced medical datasets and evaluated across a clinically reflective OOD spectrum. Our framework comprises three key components: (1) a Nonlinear von Mises‑Fisher (NvMF) classifier capable of learning non‑linear decision boundaries, with theoretical proof of its asymptotic connection to cosine classifiers; (2) a multi‑expert framework in which margin‑aware NvMF classifiers specialise in different regions of label distribution to better handle imbalance; and (3) an outlier expert trained explicitly to distinguish inlier from outlier data, thereby strengthening OOD detection. Evaluation on RFMiD, ISIC2019, and NCTCRC datasets demonstrates consistent improvements over state‑of‑the‑art methods, achieving mean FPR95 reductions of 8.45%, 13.02%, and 36.90% respectively. These gains are further supported by comprehensive ablations that validated the contributions of each component. This enables reliable identification of unfamiliar cases for deferral to clinicians, supporting safer AI‑assisted diagnosis in real‑world workflows. Our code is available at https://github.com/redboxup/MARVEL.
Authors:Wen Ying, Seongkook Heo
Abstract:
Virtual reality (VR) systems can enable convenient hand‑based interactions across diverse work scenarios. However, mid‑air gestures lack tactile feedback and a physical reference surface to support the hand. This absence of haptic grounding can cause significant challenges in achieving precise and efficient touch interactions. This paper investigates the effect of different types of hand‑grounded haptic feedback on the touch performance of VR tasks that demand high precision, such as selecting, tracing, and sketching. We compared three levels of haptic feedback: 1) No Haptic Feedback, where only visual feedback was provided; 2) Tactile Feedback, where users received vibrotactile and pressure feedback upon touching a virtual surface; 3) Physical Surface, where users interacted with a portable and tangible surface. Our study found that portable physical surfaces enabled the best selection precision, tracing efficiency, and sketch quality. Furthermore, participants showed increased bimanual hand utilization when engaging with a physical surface during tasks. These observed behaviors corresponded to participants' preference for interacting with physical surfaces, attributed to a better sense of confidence and control.
Authors:Francesca Pistilli, Simone Alberto Peirone, Giuseppe Averta
Abstract:
Understanding human behavior while interacting with the surrounding world is crucial for many applications of embodied AI. First‑person videos are particularly informative for this problem, as they well capture how activities reshape the scene over time. However, existing approaches often rely on implicit visual or language‑aligned representations, disregarding structured reasoning over the scene dynamic. We argue that explicit, compositional and editable representations of human‑environment interactions can play a crucial role for rich grounded activity understanding. To this end, we introduce SG‑Ego, a large scale annotation set extending Ego4D with spatio‑temporal scene graphs, where relations triplets are consolidated over time into explicit time‑evolving descriptions of the scene state. To reason over this representation, we propose GLEN, a graph‑based model that operates over scene graph sequences to both align them with textual actions and model their temporal evolution. In addition, we formulate the activity‑driven graph‑edit forecasting (A‑GEF) problem, a novel task that casts scene dynamics as a sequence of structured transformations conditioned on ongoing actions, enabling explicit reasoning about how scenes change over time. We validate our approach across multiple downstream tasks, spanning retrieval benchmarks as EgoMCQ and EgoCVR, as well as long‑horizon reasoning benchmarks as EXPLORE‑Bench and the newly introduced A‑GEF. GLEN achieves strong results compared to raw video baselines and it excels in reasoning settings, typically addressed only with MLLMs, while enabling controllable and structured predictions of scene dynamics driven by human activities. We believe our results establish spatio‑temporal scene graphs, together with models that reason over them, as strong compositional and interpretable representations for video understanding and potentially beyond.
Authors:Gawon Seo, Dongwon Kim, Suha Kwak
Abstract:
Decision‑time planning with action‑conditioned world models has become a popular paradigm for embodied control. However, the standard planning cost judges a candidate solely by how close its predicted terminal state lies to the goal, leaving the realizability of the intermediate transitions unchecked ‑‑ a predicted trajectory can look convincing while the environment rollout drifts away from it. In this paper, we propose ACID, a decision‑time planning framework that introduces cycle action consistency: the action inferred backward from a predicted transition by an inverse dynamics model should recover the one that was conditioned on. We fold this per‑step residual into the planning cost via a scale‑invariant adaptive weight. Across four action‑conditioned world models and six tasks spanning rigid and deformable manipulation, articulated control, and visual navigation, ACID consistently improves planning and matches the baseline's accuracy with substantially less planning compute.
Authors:Nick Stracke, Kolja Bauer, Stefan Andreas Baumann, Miguel Angel Bautista, Josh Susskind, Björn Ommer
Abstract:
Vision‑language models (VLMs) can follow complex textual instructions, yet they struggle to reason from purely visual context. In particular, current models fail to infer shared concepts from sets of example images and apply them to new inputs. We introduce Visual Concept Inference from Sets (VICIS), a task that evaluates this capability. Given a small context set of images sharing a concept and a query image, the model must generate new images that preserve the context‑defined concept while remaining consistent with the query. We show that state‑of‑the‑art VLMs perform poorly on this task, often ignoring the visual context or defaulting to biased generations. To address this gap, we propose a training framework and architecture that learn to infer visual concepts from image sets and extract concept‑specific embeddings from queries. Experiments on synthetic data and large‑scale ImageNet/WordNet data show that our model generates more accurate and diverse outputs and generalizes to unseen concepts and modalities such as sketches.
Authors:Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-Martínez
Abstract:
Large Language Model (LLM) inference workloads are a rapidly growing contributor to data center energy consumption. Optimizing these deployments requires matching specific LLMs to the most efficient GPUs, but operators currently lack the tools to do so without exhaustively profiling each combination. While some predictive models exist, they still require profiling data and struggle to generalize to hardware unseen during training. To address this, we introduce WattGPU, featuring two predictive models for mean GPU power draw and Inter‑Token Latency (ITL). Our approach leverages only publicly available LLM metadata and GPU specifications, eliminating the need for hardware access or profiling while enabling generalization to unseen NVIDIA server‑grade GPUs and LLMs. We evaluate our models using rigorous leave‑one‑GPU‑out and leave‑one‑LLM‑out cross‑validation on a dataset of 42 open‑source LLMs (0.1B‑‑27B parameters) and 8 GPUs under both offline and server scenarios. The mean power draw model achieves a median absolute percentage error of \leq3.4% for offline and \leq13.5% for server scenarios on unseen GPUs, while the latency model achieves \leq8.5% in server mode, both maintaining strong GPU ranking correlations for server scenarios (Kendall τ\geq0.76). Compared to standard physically grounded baselines ‑‑ Load‑Scaled Thermal Design Power (TDP) for power draw and roofline for latency ‑‑ our models reduce median absolute percentage error by approximately 4× on unseen LLM‑GPU combinations for server scenarios or approximately 2× for completely unseen GPUs. WattGPU's data and code are publicly available at https://github.com/maufadel/wattgpu.
Authors:Benjamin Nichols, Michael Schlichtkrull, Nedjma Ousidhoum
Abstract:
LLM‑based retrieval‑augmented generation (RAG) is increasingly used for automated fact‑checking (AFC) and related tasks. By grounding LLM outputs in retrieved evidence, RAG‑based systems provide transparent justifications while allowing external information to be updated independently of the underlying model. However, existing approaches often assume retrieved evidence is reliable, although real‑world information may be conflicting, outdated, and can originate from unreliable or biased sources. Recent work on source‑critical reasoning addresses this challenge through media background checks (MBCs) (Schlichtkrull, 2024), which assess the credibility of evidence sources to support downstream fact verification. However, generating MBCs relies on costly proprietary search APIs, limiting reproducibility. To mitigate this issue, we introduce MEDIAREF, a publicly available knowledge store of web‑sourced documents that enables reproducible, low‑cost evaluation of MBC generation across 200 media sources. We describe a reproducible methodology for constructing and updating the collection, assess widely used LLMs on the MBC generation task, and demonstrate that MEDIAREF supports higher‑quality MBC generation through both automatic and qualitative evaluation.
Authors:Lan Feng, Wuyang Li, Eloi Zablocki, Matthieu Cord, Alexandre Alahi
Abstract:
We elucidate the design space of Representation Distribution Matching (RDM), our name for the paradigm that trains a one‑step image generator by matching generated and reference feature distributions under frozen pretrained encoders. We identify two design axes, how the distributions are compared and the representations they are compared in, and controlled studies along them yield three findings. First, the classical MMD, which could not train convincing generators a decade ago, becomes a strong and scalable objective once estimated right. Second, the generated batch is then the operative variable, with an optimum above 2048, far beyond customary batch sizes. Third, any single representation can be gamed, driven below the real score while images stay visibly fake, so we match against a balanced battery of encoders and evaluate with SW_r14, a Sliced‑Wasserstein distance over 14 encoders that is independent of the training loss and resists gaming. Combining the preferred choices yields improved RDM (iRDM): it sets the one‑step state of the art on ImageNet at SW_r14 1.30, corroborated by PickScore, a human‑preference proxy our objective never optimizes, which prefers it over the prior best one‑step generator on 71.2% of matched samples. The same recipe post‑trains the four‑step FLUX.2 [klein] into a one‑step generator, surpassing the four‑step version on GenEval, 0.826 to 0.794, and on PickScore, 22.76 to 22.58, in 90 H200 GPU‑hours. Project page: https://alan‑lanfeng.github.io/rdm/.
Authors:Polina Karpikova, Wenjing Bian, Haofei Xu, Hendrik Lensch, Andreas Geiger
Abstract:
Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization‑based methods achieve high fidelity but require costly per‑scene fitting, while image‑space learning‑based approaches often suffer from multi‑view inconsistencies and lack an explicit 3D representation for stable novel view rendering. We present a feed‑forward multi‑view reconstruction framework for inverse rendering that directly predicts a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, enabling a disentangled and physically grounded scene representation. Our model integrates priors from a material estimation network with a multi‑view 3D reconstruction backbone, allowing joint prediction of geometry and reflectance parameters in a single forward pass. Experiments on synthetic and real‑world datasets demonstrate improved multi‑view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. Our representation further supports physically‑based relighting and more faithful modeling of view‑dependent effects compared to existing RGB‑based feed‑forward reconstruction methods. Our project webpage is: \hrefhttps://poliik.github.io/invsplat/\texthttps://poliik.github.io/invsplat/.
Authors:Haiyang Li, Yuming Fu, Qun Song, Hongchao Liao, Jing Chen, Mounim A. EI-Yacoubi, Xin Jin
Abstract:
Vein recognition is a secure biometric technology often constrained by limited annotated data and imaging variations. While data augmentation mitigates this, strategies designed for natural images may disrupt the fine‑grained topology and textures essential for identity discrimination. We present AGVBench, which evaluates 30 representative augmentation strategies on five public palm‑ and finger‑vein datasets with seven backbone architectures, covering classic CNNs, vision transformers, and vein‑specific recognition models. Our results show that multi‑image mixing methods (e.g., MixUp, PuzzleMix, StarMixup) generally provide the strongest recognition performance. However, they are often poorly calibrated and vulnerable to adversarial perturbations, revealing a clear inconsistency between clean accuracy and adversarial security. We also find that severe geometric transformations frequently degrade recognition, which is potentially due to feature misalignment or spatial cropping, and that augmentation effectiveness varies across palm and finger vein datasets. These findings prove that accuracy‑centric evaluation is insufficient for biometric augmentation. AGVBench provides standardized protocols to support reproducible research and guide the design of reliable, secure, and robust vein recognition systems. Our codebase is available at https://github.com/Advance‑VeinTech‑Innovators/AGVBench.
Authors:Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang, Daren Zha, Shi Yu, Zhenghao Liu, Shuo Wang, Xu Han, Maosong Sun
Abstract:
Reasoning Language Models (RLMs) have significantly improved performance on complex tasks by extending the reasoning chain. However, these chains are prone to containing factual errors, particularly in knowledge‑intensive tasks. To address this issue, we propose CheckRLM, a framework that improves the reliability of the reasoning process through Retrieval‑Augmented Generation (RAG) by timely checking and correcting factual errors. Specifically, CheckRLM extracts factual claims from the reasoning chain to identify and localize subtle knowledge inconsistencies during inference. Upon detection of errors, a refinement mechanism performs minimal‑cost yet precise corrections by leveraging external knowledge, ensuring coherence between the reasoning chain and correct knowledge. Extensive experiments demonstrate that CheckRLM substantially outperforms existing baselines, exhibiting a strong capability to mitigate error accumulation in long‑horizon reasoning with lower costs. The code and data are available at https://github.com/AI9Stars/CheckRLM.
Authors:Ningning Han, Lei Fan, Jia Guo, Yunkang Cao, Xiu Su, Feng Cao, Donglin Di, Tonghua Su
Abstract:
The deployment of Industrial Anomaly Detection (IAD) in real‑world manufacturing frequently encounters a challenging cold‑start bottleneck, in which limited normal samples fail to represent the full normal distribution and only a few anomalies are available. Under such a regime, existing methods struggle to form compact normal boundaries and fail to effectively exploit supervised signals from rare defects. To address this challenge, we propose Anomaly‑Rectified Cold‑start AD (ArcAD), a plug‑and‑play calibration framework for reconstruction‑based IAD baselines. ArcAD follows a push‑pull learning paradigm to construct a compact and discriminative normal boundary under data scarcity. On the one hand, ArcAD projects limited normal samples onto a hypersphere and pulls them into multiple compact clusters to maximize coverage of the normal manifold. On the other hand, it synthesizes pseudo‑anomalies on the hypersphere and leverages real anomalies to push the boundary inward and sharpen anomaly discrimination. Extensive experiments on MVTec‑AD, VisA, Real‑IAD, and MANTA demonstrate that ArcAD significantly outperforms state‑of‑the‑art supervised and unsupervised methods in both single‑class and multi‑class settings under cold‑start conditions. Code is available at: https://github.com/LGC‑AD/ArcAD.
Authors:Tien-Phat Nguyen, Ngai-Man Cheung
Abstract:
Vision Transformers (ViTs) are strong backbones for semantic segmentation, but their computational cost limits deployment. Recent token compression methods for efficient transformer‑based segmentation reduce this cost by decreasing the number of tokens. However, existing evaluations primarily focus on low‑to‑moderate compression, leaving their behavior under aggressive compression and corrupted inputs unclear. Meanwhile, structural pruning provides an orthogonal route to efficiency by removing redundant components in the ViT architecture, but is rarely compared to token compression under a unified protocol. To bridge this gap, we benchmark representative token compression and structural pruning methods for ViT‑based semantic segmentation under matched FLOPs on ADE20K and Cityscapes, together with their common‑corruption variants ADE20K‑C and Cityscapes‑C. Our results reveal a consistent trend on both clean and corrupted inputs: token compression is highly effective at mild reductions but degrades sharply when compression becomes severe, consistent with substantial information loss from overly aggressive token reduction. In contrast, structural pruning exhibits a smoother degradation curve and is more stable at high compression. Motivated by these findings, we study a prune‑then‑merge pipeline that applies moderate token compression on top of a moderately pruned backbone. At comparable FLOPs, this combined strategy consistently achieves a better accuracy‑robustness trade‑off at high compression, offering a practical recipe for deployment‑oriented ViT segmentation. Code is available at https://github.com/phatnguyencs/vit‑seg‑compression.
Authors:Yongjie Bai, Hanting Wang, Mingtong Dai, Qijun Zhong, Yang Liu, Liang Lin
Abstract:
General‑purpose vision‑language‑action models benefit from large vision‑language priors, but effective manipulation also requires anticipating action‑relevant scene changes. Existing world‑action models often rely on large generative world models or dense future rollouts, which are expensive and spend capacity on visual details weakly coupled to control. We present Bridge‑WA, a lightweight world‑action framework that distills a frozen future‑change teacher into three compact priors: future tokens for intended outcomes, change maps for intervention support, and motion‑flow maps for local transition direction. A WorldBridge conditions the action transformer on these priors through multi‑source attention memories and spatial‑temporal biases, while the teacher model is removed at inference. Across VLABench, RoboTwin2.0, LIBERO‑Plus and real‑robot evaluations, Bridge‑WA improves task success, progress, and robustness, with particularly clear gains under out‑of‑distribution visual shifts. By focusing action generation on where and how the scene will change, Bridge‑WA suppresses nuisance appearance factors such as background, lighting, and distractors, leading to better generalization without deployment‑time dense future‑image generation. Code and visualizations are available at: https://hcplab‑sysu.github.io/BRIDGE‑WA .
Authors:Varshith Roy Kotla
Abstract:
Conventional traction control architectures intervene only after the adhesion limit of a tire has already been breached. This paper investigates whether Rolling Split Conformal Prediction , monitoring the volatility of non‑conformity residuals from a per‑driver Random Forest model of expected slip behavior , can serve as a statistically grounded pre‑incident warning signal, ahead of gross traction loss. Unlike an earlier internal draft of this work, the evaluation reported here corrects a confound in the slip proxy (vehicle speed is included as an explicit model feature, not left implicit in the target's denominator), uses every racing lap for each driver rather than only the fastest lap, and is scored against real, timestamped incident labels extracted from FIA Race Control Messages and track‑limits lap deletions rather than narrated post‑hoc. The result is negative: across 19 drivers and 55,563 test‑phase telemetry samples, the rolling‑volatility detector achieves a mean precision of essentially 0.0 and mean recall of 0.0 against 14 ground‑truth incidents, while flagging on average 15.3% of all samples as anomalous , too high a false‑alarm rate for any early‑warning use. A static 95th‑percentile threshold baseline performs no better in any way that would justify the added complexity of the conformal‑volatility formulation. Residual autocorrelation diagnostics show the split‑conformal exchangeability assumption is violated for every driver (Ljung‑Box p < 0.001, n = 19/19), which is one plausible driver of the high false‑alarm rate. We report this as a methodologically rigorous negative finding, diagnose its likely causes, and outline what a genuinely predictive version of this approach would require.
Authors:Tomasz Szczepański, Szymon Płotka, Michal K. Grzeszczyk, Tomasz Trzciński, Arkadiusz Sitek
Abstract:
Generating a 3D dental volume from a single panoramic radiograph (PXR) could provide a low‑radiation alternative to Cone‑Beam Computed Tomography (CBCT), but the problem is highly underdetermined: panoramic acquisition integrates 3D attenuation along curved X‑ray paths into a 2D image, leaving depth‑resolved anatomy unobserved. Existing implicit and generative approaches often produce oversmoothed geometry or anatomically inconsistent hallucinations, lacking geometry‑driven supervision and relying on smooth representations unable to precisely localize sharp anatomical boundaries. We propose X‑Splat, the first Gaussian Splatting framework for generating CBCT‑like 3D dental volumes from a single PXR. X‑Splat uses the known panoramic acquisition geometry as a generation scaffold: learnable anisotropic Gaussian primitives are initialized along the X‑ray paths that formed the input image and adjusted in a single feed‑forward pass, constrained by Beer‑Lambert reprojection and multi‑view radiographic training supervision. A lightweight residual refiner adds dataset‑level anatomical priors without overriding the geometry already resolved by the Gaussians. We train on synthetic PXR‑CBCT pairs, enabling direct volumetric supervision without paired real scans. We further introduce segmentation‑based geometry‑aware metrics, providing the first evaluation of PXR‑based generation over maxillofacial anatomy. X‑Splat outperforms NeRF‑ and GAN‑based baselines, recovering individual teeth, cortical boundaries, and alveolar structure, including the mandibular canal which prior methods fail to reconstruct. Code will be available at https://github.com/tomek1911/X‑Splat
Authors:Wan Song, Wei Zhou, Rui Wang, Jun Yu, Toru Kurihara, Jiajia Xu, Shu Zhan
Abstract:
Large kernel depthwise convolutions achieve strong performance but suffer from significant degradation as kernel size grows due to irregular memory access from gather‑based computation; while Large Kernel Acceleration (LKA) helps on small feature maps, it becomes counterproductive on large feature maps, even slower than non‑accelerated implementations. We propose Windowed Batch Matrix Multiplication (WBMM), which partitions input into contiguous windows and indexes a compact relative position bias table to construct weight matrices, enabling regular memory access via batched matrix multiplication. This yields a unique property: WBMM's throughput improves with larger windows, opposite to depthwise convolutions that degrade with larger kernels. Operator‑level benchmarks show WBMM with 14x14 windows outperforms 5x5 depthwise convolution baselines in speed while providing a 7.8x larger per‑layer receptive field. Combined with inter‑block cross‑window communication and hierarchical window reparameterization, WBMM achieves comparable or higher accuracy on ImageNet‑1K, COCO, and ADE20K with 1.31‑1.88x training speedup, and demonstrates consistent advantages across GPU, CPU, and edge devices without requiring specialized acceleration kernels. Our code is available at http://github.com/wansong‑s/WBMM
Authors:Shunya Kato, Taiki Miyanishi, Shuhei Kurita, Mahiro Ukai, Nakamasa Inoue, Chenhui Chu
Abstract:
Egocentric videos capture rich and diverse human‑object interactions and have emerged as a fundamental resource for understanding human activities related to objects. In this context, Video Referring Expression Comprehension (Video REC), the task of localizing the temporal and spatial extent of a referred object in video frames given a natural language query, plays a key role in linking textual descriptions to observed objects in untrimmed egocentric recordings. However, existing egocentric Video REC benchmarks primarily focus on short video clips, where some target object appears densely within frames. Such settings do not reflect real‑world egocentric recordings, which are long‑form, untrimmed, and characterized by sparse object occurrences and complex activity transitions. To address this limitation, we introduce LongEgoRefer, a novel and challenging benchmark constructed from long‑form videos in the Ego4D dataset. LongEgoRefer contains 1,498 referring expressions with an average video duration of 45 minutes. The benchmark exhibits extreme target sparsity, detailed linguistic descriptions, and complex human‑object interactions embedded in long, dynamic egocentric narratives. Consequently, it defines a demanding spatio‑temporal grounding problem that requires models to identify both when an event occurs and where the referred object appears within extended video sequences. We evaluate existing Video REC approaches, including training‑free baselines based on vision‑language models combined with Grounded SAM2. Extensive experiments show that even advanced baselines and current state‑of‑the‑art models struggle significantly on LongEgoRefer. These results highlight the intrinsic difficulty of long‑form egocentric spatio‑temporal grounding and emphasize the need for more robust video understanding models.
Authors:Tien-Huy Nguyen, Minh-Nhat Nguyen, Nguyen Nhat Huy, Hung Viet Nguyen, Huy Nguyen Minh Nhat, Thanh-Huy Nguyen, Cuong Tuan Nguyen, Hoang M. Le, Dat Nguyen, Phat Kim Huynh, Min Xu, Ulas Bagci
Abstract:
Vision‑language models (VLMs) have achieved strong performance across diverse multimodal tasks, yet they remain vulnerable to unreliable reasoning. Existing self‑correction methods mitigate these issues but typically rely on post‑training or carefully engineered feedback, incurring high computational cost. In this work, we revisit this challenge through the lens of emotional cues, asking whether they can activate latent self‑correction behaviors in VLMs without additional training. We find that emotional signals serve as an effective trigger for self‑correction, encouraging more cautious and reflective reasoning. Motivated by this finding, we propose \escabstract (\underlineEmotional \underlineSelf‑\underlineCorrection), a training‑free self‑correction framework. ESC introduces an external verifier that detects potentially incorrect initial responses and injects emotional feedback to encourage model to reflect, and produce a better revised response without additional training. Extensive experiments across safety, hallucination, vision‑centric perception, and multimodal reasoning benchmarks show that ESC consistently improves reliability while preserving overall model utility. These results suggest that emotion can function not only as an ability to be recognized, but also as a practical control signal for scalable self‑correction in VLMs. We therefore believe that ESC provides a strong foundation for a new reliable human‑like, emotion‑integrated research direction. Our project is publicly available at \textcolorredhttps://genai4e.github.io/ESC/.
Authors:Oren E. Livne
Abstract:
We present NLF (Nonlinear Laplacian Flow), a unified framework and linear‑time solver for convex network‑flow equilibria. Congestion routing, minimum‑delay routing, and maximum flow share one form: the nonlinear graph Laplacian Bρ(B^Tϕ)=αd, where a monotone edge law ρ_e encodes the physics (undirected graphs; directed variants are future work). NLF solves it by a damped chord‑Newton iteration whose frozen linearization ‑‑ a weighted graph Laplacian ‑‑ is inverted by a near‑linear Laplacian solver (default: approximate Cholesky, LAMG+ interchangeable). The nonlinear solve costs 2‑‑4 linear Laplacian solves, making the wall‑clock empirically O(m) in the edge count m (not a proved bound). On single‑commodity congestion (BPR cost), NLF converges on all 2,003 SuiteSparse corpus graphs up to 1.8×10^7 edges. Against a state‑of‑the‑art interior‑point method, NLF is a median 2.6× faster where both converge and >45× on poorly‑separable graphs where the IPM's direct core is superlinear; against L‑BFGS, a median 4.2× faster and the only solver to finish on the 90 hardest instances. A multicommodity extension routes K commodities through one shared hierarchy at O(Km) per step. The same machinery recovers the exact max‑flow as a short sequence of Laplacian solves, with the cut potential as a by‑product. Code: https://github.com/orenlivne/nlf
Authors:Lu Pan, Hongwei Zhao
Abstract:
Physics‑based Human‑Scene Interaction (HSI) imitation learning is crucial for embodied intelligence as it bridges the gap between kinematic 3D motions and real‑world dynamics. However, most existing methods focus on simplified scene settings, leaving complex environments largely unexplored, which limits their applicability in real‑world scenarios. In this paper, we focus on HSI mimicry in complex environments. Under this complex setting, we observe an inherent trade‑off between successfully performing interaction and maintaining natural, physically plausible motions. To address this challenge, we propose ComplexMimic, a framework that reconstructs diverse HSI by interpreting imperfect MoCap data. First, we introduce a Dual Flow Strategy, which learns two complementary experts: an imitation expert for accurate motion tracking and an interaction expert for collision‑aware adaptation in complex scenes. Second, naive multi‑expert distillation, which treats all experts equally, often under‑samples challenging behaviors, limiting effective learning. To mitigate this issue, we propose a difficulty‑aware distillation strategy that adaptively weights supervision and prioritizes hard‑yet‑learnable trajectories guided by failure statistics and learning progress signals. Extensive experiments on three benchmark datasets demonstrate that our approach outperforms current state‑of‑the‑art methods. Our implementation is available at https://github.com/LuPan23/ComplexMimic.
Authors:Xiaopei Zhu, Zeyuan Li, Jun Zhu, Xiaolin Hu
Abstract:
Mirror Illusion Art is a novel reflection‑conditioned 3D illusion where one object yields two target appearances (front and mirror). The task is formulated as inverse design from two target 2D images (front and mirror) to a printable 3D object with geometry and texture. Prior topology‑driven and shadow‑based approaches demand substantial manual effort, optimize shape only, and often yield non‑smooth or incomplete geometry. To address these challenges, we propose AutoMIA, an automated Mirror Illusion Art design pipeline that jointly optimizes shape and color. To stabilize optimization and suppress artifacts, four mechanisms are introduced: (1) projection‑alignment component (PAC) selection to reduce surface noise, (2) position‑weighted adaptive (PWA) suppression for background noise, (3) internal voxel preservation (IVP) to prevent internal fractures, and (4) shape‑color decoupled (SCD) optimization that balance shape and color optimization. AutoMIA generate diverse smooth Mirror Illusion artworks successfully both in the digital and physical world, with only around 76s design time and 2.6 GB memory on average using a single RTX 3090, advancing inverse graphics and computational design. Our code is available at https://github.com/zxp555/AutoMIA.
Authors:Weichen Zhou, Yawen Zou, Chunzhi Gu, Ran Dong, Haoran Xie, Chao Zhang
Abstract:
We introduce a controlled subspace intervention framework to investigate how self‑supervised Vision Transformers (ViTs) encode dense geometric information. While linear probing is widely used to assess geometric representations, it treats features as a black box, failing to disentangle the underlying topology. To address this issue, we decompose the weights of converged linear probes to isolate the low‑rank subspaces containing explicit geometric signals using Singular Value Decomposition (SVD). Our perspective yields three key insights: (1) Pre‑training objectives determine how features are encoded. DINOv2 aligns spatial features for efficient linear extraction, while Masked Autoencoders (MAE) tend to disperse these signals, requiring a broader spatial context. (2) Explicit geometric representations are highly compressible, suggesting dense predictive heads could potentially be constrained to low‑rank subspaces with minimal performance loss. (3) The layer‑wise task affinity suggests that geometric precision peaks at intermediate layers before yielding to semantic abstraction in the final layers. By connecting internal encoding mechanics with downstream performance, these findings provide a basis for effective feature selection and lightweight decoder design. The source code is available at https://github.com/Zhou‑Weichen/Geosubprobe.
Authors:Siyuan Li, Youyuan Zhang, Ruitong Liu, Junxi Wang, Jing Li
Abstract:
Online multimodal knowledge editing requires injecting a continual stream of visual‑textual corrections into multimodal large language models (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors. Existing editors mainly emphasize edit reliability and long‑horizon stability, but rarely control the semantic boundary of each edit. Our pilot analyses of post‑edit behaviors and internal neuronal activities reveal a scope gap behind reliable edits: instance‑level success neither guarantees transfer to valid cross‑modal variants nor prevents leakage to unrelated inputs, while edit‑related cross‑modal responses concentrate in deeper semantic layers. Therefore, we formulate Edit‑Scoped Generalization, reframing online MLLM editing from merely correcting an instance to controlling the propagation boundary of each edit. To this end, we propose ScopeEdit, a scope‑aware online editor that decomposes each update into a modality‑local absorption branch and an evidence‑gated shared generalization branch. The local branch supports stable edit absorption, whereas the shared branch enables cross‑modal propagation only when visual and textual evidence are sufficiently aligned. Both branches perform scope‑separated write geometries in orthogonal low‑rank spaces and maintain branch‑wise preconditioners via Sherman‑‑Morrison recursions, yielding constant per‑edit overhead. Extensive experiments across diverse benchmarks, long‑horizon edit streams, MLLM backbones, real‑world VLKEB scenarios, and complex vision‑language architectures show that ScopeEdit consistently improves the trade‑off between in‑scope cross‑modal transfer and out‑of‑scope locality, while preserving edit reliability, stability and online efficiency. Our code is available at https://github.com/lab‑klc/ScopeEdit.
Authors:Hamed Babaei Giglou, Jennifer D'Souza, Andrei Aioanei, Nandana Mihindukulasooriya, Sören Auer
Abstract:
Ontology learning (OL) aims to automatically construct structured knowledge models from text, yet progress remains fragmented across methods, domains, and evaluation practices. Despite decades of research, OL lacks a shared infrastructure for systematic evaluation and ontology access. This absence has hindered progress and fragmented research, leaving the central challenges of OL largely unaddressed. We introduce OntoLearner, a modular, cross‑domain, and first‑of‑its‑kind framework that unifies ontology access, large language model (LLM)‑driven learning pipelines, and standardized benchmarking. OntoLearner releases 180 machine‑readable ontologies spanning 22 domains and provides pipeline‑ready datasets with train/dev/test splits for three core OL tasks: term typing, taxonomy discovery, and non‑taxonomic relation extraction. Using this infrastructure, we conduct a large‑scale empirical study of OL, evaluating 22 retrieval models and 12 LLMs across domains and tasks. The results converge on a finding that reframes the central challenge of OL: failure modes scale with ontological complexity rather than model size or architectural sophistication. The primary bottleneck is not model capability, but a structural mismatch between how models encode knowledge and how ontologies organize it. These findings establish that effective OL is reachable through the cross‑domain, multi‑task benchmarking enabled by OntoLearner. OntoLearner is open‑source (MIT license) at https://github.com/sciknoworg/OntoLearner/.
Authors:Jinxi Li, Tianyi Zhang, Yafei Yang, Zihui Zhang, Peng Huang, Koon Wing Macgyver Lin, Bo Yang
Abstract:
We study the challenging problem of novel view video synthesis from single images or monocular videos. Existing methods, which operate under the assumption that pre‑trained video models lack native novel view synthesis capability and enforce view alignment via camera conditioning, task‑specific fine‑tuning, or stepwise hard denoising guidance, often suffer from artifacts and compromised global scene consistency. In this paper, we introduce NeoMap, a novel training‑free framework designed to locate high‑fidelity, view‑consistent novel view solutions from general pre‑trained video models. The key to our approach is the core insight that promising novel view solutions are inherently encoded within the natural video data manifold learned by pre‑trained models, and the core challenge is simply to locate this optimal solution. We solve this via our core mechanism: convergent manifold alternating projection iterations that optimize the initial noise. Extensive experiments demonstrate that NeoMap significantly outperforms all existing methods across 3 standard novel view synthesis benchmarks, including the challenging Tanks‑and‑Temples, LLFF and DAVIS datasets, achieving state‑of‑the‑art generation fidelity and top‑tier view consistency.
Authors:Uzair Khan, Luigi Capogrosso, Muhammad Aqeel, Francesco Setti, Michele Magno, Marco Cristani
Abstract:
In modern high‑throughput industrial production lines, product configurations and visual characteristics frequently change, making it impractical to collect and annotate data for every new scenario. This dynamic setting makes Zero‑Shot Anomaly Detection (ZSAD) particularly suitable, as it enables defect detection without requiring training on target‑specific samples. Although recent ZSAD approaches show promising results, they are computationally intensive and thus unsuitable for deployment on resource‑constrained devices. We propose LiZAD: a lightweight framework designed for real‑time ZSAD specifically tailored for use on edge devices. The proposed approach pairs the dense and spatially aware visual features of DINOv3, crucial for precise pixel‑level localization, with the highly computationally efficient text embeddings of MobileCLIP2. These features are then mapped into a shared latent space via low‑memory trainable projection heads. Compared to six state‑of‑the‑art ZSAD models, LiZAD achieves an average memory reduction of 61.5%, a parameter reduction of 74.6%, and a speedup of 3.02x in terms of latency. Despite substantial reductions in computational and memory costs, our approach maintains competitive anomaly detection performance, dropping the average P‑AUROC by just 6.4% relative to the best state‑of‑the‑art model across the VisA, BTAD, MPDD, and MVTec‑AD datasets. Finally, it is successfully deployed on the NVIDIA Jetson NX and Jetson AGX edge devices and tested on the real production line of the Industrial Computer Engineering Laboratory (ICE Lab) at the University of Verona. The code is available at https://github.com/intelligolabs/LiZAD.
Authors:Peng Yun, Shouwang Huang, Hao Li, Jinxi Li, Jianan Wang, Bo Yang
Abstract:
Manipulating fast and dynamically moving targets in unstructured 3D environments remains challenging for embodied AI. Existing visual‑language‑action models and world models struggle with accurate 3D geometry and physically meaningful forecasting. We propose PhysMani, a framework that couples a physics‑principled 3D Gaussian world model with a future‑aware action policy model. The world model learns a divergence‑free Gaussian velocity field via online optimization for fast and physically grounded future dynamics prediction. The policy model integrates the predicted 3D scene future dynamics through a learnable token based cross‑attention module. We introduce PhysMani‑Bench, a dynamic manipulation benchmark with 16 tasks, and demonstrate a superior success rate over strong baselines in both simulation and real‑world robot experiments.
Authors:Zixuan Chen, Hao Fu, Haiwen Hu, Shiquan Zheng
Abstract:
Offline reinforcement learning (RL) holds significant potential for crowd robot navigation in human‑robot coexistence applications. However, the inherent complexity of pedestrian motion renders the design of effective reward functions for promoting socially compliant robot behaviors a persistent challenge. This paper proposes a Social Preference Learning for Crowd Robot Navigation (SPLC) algorithm to eliminate the need for detailed reward design. Its core innovation lies in the introduction of a social preference feedback mechanism to automatically generate preference data through principled preference evaluation criteria. By explicitly accounting for the intricacies of pedestrian dynamics, the pipeline mitigates the reward bias and facilitates the systematic quantification of broad social norms, thereby fostering socially compliant behaviors. Extensive experiments integrating SPLC with offline RL methods demonstrate consistent improvements over state‑of‑the‑art baselines across standard performance metrics. Furthermore, real‑world experiments on the TurtleBot4 further validate the effectiveness of SPLC in practical human‑robot coexistence settings. Our code and video demos are available at https://github.com/sklus949/SPLC.
Authors:Chiwang Luk, Matin Mohammad Najafi, Zhifeng Jia, Wei Yang, Xiuchang Li, Jinwei Zhu, Yang Ren, Lei Chen, Gao Cong
Abstract:
Large language model agents can repair real repository issues, but they often spend large context budgets on whole‑file reads, broad searches, and long terminal outputs where useful evidence is mixed with irrelevant code and logs. This paper presents ContextSniper, AntTrail's token‑efficient code memory layer for repository‑level program repair. As the coding specialization of AntTrail's broader agent memory engine, ContextSniper implements the Sniper feature for precision evidence selection: it retrieves candidate code and runtime evidence, ranks it with hybrid retrieval signals, filters long outputs through an intention‑aware context gate, and returns compact evidence packets while preserving recoverable source context outside the prompt. We evaluate ContextSniper on SWE‑bench Lite with OpenClaw and Claude Code, using 50 task runs per host‑agent condition. ContextSniper reduces total token use by 51.5% and logged cost by 36.4% for OpenClaw, and reduces total token use by 38.9% and estimated cost by 27.3% for Claude Code. Submitted‑resolution rates decrease slightly, from 26.0% to 24.0% for OpenClaw and from 32.0% to 30.0% for Claude Code. ContextSniper's pilot testing scripts are open‑sourced at https://github.com/Calluking/ContextSniper
Authors:Bingcong Yan, Chunlei Li, Jingliang Hu, Yilei Shi, Xiao Xiang Zhu, Lichao Mou
Abstract:
Large vision‑language models (LVLMs) have achieved strong performance across many medical imaging tasks, yet their application to ultrasound remains limited due to its inherent complexity and variability. In this work, we revisit what is truly needed to enable real‑world ultrasound understanding. Instead of introducing complex architectures or elaborate training strategies, we show that data scale and clinically faithful data alignment are the key factors. We construct a large‑scale dataset of 1.5M real‑world ultrasound examinations, containing 17.7M images, multi‑organ coverage, and paired uncurated clinical reports. Crucially, we organize the data at the examination level, aligning multiple images with their corresponding reports to reflect real clinical workflows. We then fine‑tune a standard LVLM using low‑rank adaptation (LoRA) on this dataset without task‑specific modifications. Surprisingly, this simple recipe already leads to strong performance across diverse ultrasound understanding tasks, outperforming prior methods designed with more complex pipelines. Beyond these results, we present model and data scaling analyses that provide insights into the role of scale in ultrasound LVLMs.
Authors:Fengchen He, Hao Xu, Dayang Zhao, Tingwei Quan, Shaoqun Zeng
Abstract:
Dual‑pixel (DP) imaging enables metric depth estimation from a single camera using sub‑aperture disparity. However, the extremely small effective baseline limits disparity observability, leading to structural degradation and depth failure in textureless, low‑contrast, or downsampled regions. Existing DP‑based methods rely primarily on local disparity cues and therefore become unreliable when disparity signals are weak or ambiguous. To address this limitation, we propose \emphFoundDP, a unified framework that integrates metric DP depth with global structural priors from a monocular depth foundation model. Our method preserves metric scale through DP‑derived depth and leverages Vision Transformer (ViT) features to restore structural consistency in weak‑disparity regions. To ensure reliable metric guidance under DP imaging conditions, we identify and mitigate ViT representation degradation induced by DP defocus blur via ViT feature alignment, enabling stable metric‑guided depth estimation. Extensive experiments on synthetic and real‑world DP benchmarks show that FoundDP delivers superior performance, with consistent gains in structural fidelity and metric accuracy, especially under reduced disparity observability. Code will be available at: https://github.com/EchoLighting/FoundDP
Authors:Junhao Chen, Xiang Li, Mingjin Chen, Boran Zhang, Henghaofan Zhang, Yibin Xu, Yuehan Cui, Fangsheng Weng, Fei Ma, Qi Tian, Ruqi Huang, Hao Zhao
Abstract:
Code is the medium through which large language models generate structured artifacts: charts, scientific figures, vector graphics, CAD models, 3D scenes, and hardware designs are all produced by writing programs. In this regime single pass inference is brittle, because the compiler, renderer, or simulator that decides whether the artifact exists is invisible to the model. We present PairCoder, which grounds review in the toolchain and realizes it as two agent pair programming: a Driver agent writes the program, a Navigator agent reviews it against verification evidence (diagnostics, execution results, and renderings of the current artifact beside the target), and the two switch roles when errors persist. Across 17 public benchmarks and seven models from three vendors, PairCoder improves essentially every benchmark whose artifact is verifiable, on full official metric suites rather than execution alone (for example, Blender scene executability 0.20 to 0.78; TikZ compile rate up 10 to 30 points on every model), at 2.9 to 9.2 times single model cost (about 7 times overall). The improvements concentrate where the toolchain provides an informative oracle and the baseline leaves headroom, and the method ties or mildly regresses where the oracle is weak; we frame pair programming as a reliable recipe for verified code driven generation.
Authors:Qi Lyu, Jiahua Dong, Baichen Liu, Xudong Wang, Mingfei Han, Yulun Zhang, Fahad Shahbaz Khan, Salman Khan, Lianqing Liu, Zhi Han
Abstract:
Large Vision‑Language Models (LVLMs) have achieved remarkable progress in multimodal understanding, yet their enormous parameter scale and cross‑modal computation incur substantial memory and latency overhead, severely limiting real‑world deployment on resource‑constrained devices. Binarization offers an attractive solution by drastically reducing storage and computational costs. However, existing binarization methods neglect the varying importance of weights across different layers and modalities. This causes parameters irrelevant to downstream tasks to be unnecessarily retained, whereas modality‑critical weights may not be adequately optimized, resulting in significant performance degradation. To address these challenges, we develop a novel \underlineSignificance‑\underlineAware \underlineBinarization for \underlineLarge \underlineVision‑\underlineLanguage \underlineModels (SAB‑LVLM). Specifically, after constructing Hessian matrices for textual and visual inputs, we propose a spatial significance map to distinguish full‑precision weights activated under a single modality from those activated across modalities. We then devise a modality‑guided integration strategy to obtain the significance‑aware binarization map, which measures weight significance across layers and modalities. Subsequently, this binarization map is incorporated into the binarization objective as an error reweighting term, and binarization fitting is performed through an alternating significance‑weighted update scheme. Extensive experiments illustrate the superiority of our SAB‑LVLM over existing binary PTQ methods under an approximately 1‑bit compression constraint. Our code is accessible at https://github.com/LyuQi127/SAB_LVLM.
Authors:Dawei Ren, Yan Zhang, Hongying Tang, Qiaoling Zhou, Jianpo Liu
Abstract:
Camouflaged Object Detection (COD) aims to locate and segment objects that blend into their surroundings, presenting challenges due to weak edge cues and ill‑defined boundaries. Traditional COD models rely on hand‑designed architectures and multi‑scale feature fusion, which are often guided by intuition rather than systematic search. This paper introduces CamoNAS, a frequency‑aware multi‑resolution Neural Architecture Search (NAS) framework for COD. CamoNAS automatically searches both cell‑level operations and network‑level downsampling paths, forming a hierarchical search space tailored to detect camouflaged objects. Additionally, it adopts an RGB frequency dual‑stream architecture, where a learnable wavelet transform complements the RGB spatial stream. CamoNAS achieves state‑of‑the‑art performance on four COD benchmarks (CAMO, COD10K, NC4K, CHAMELEON), highlighting the effectiveness of NAS for COD. Our code is available at https://github.com/rendaweiSIMIT/CamoNAS.
Authors:Clémentine Grethen, Florient Chouteau, Géraldine Morin, Simone Gasparini
Abstract:
Large 3D foundation models such as MASt3R achieve state‑of‑the‑art stereo reconstruction but are computationally demanding for deployment under strict hardware constraints ‑‑ a critical limitation in domains such as planetary exploration, where onboard computing is severely restricted. We study how far such models can be compressed through knowledge distillation, using lunar stereo reconstruction as a challenging and practically relevant case study. Starting from a 688M‑parameter MASt3R teacher fine‑tuned on lunar imagery, we distill its dense geometric predictions into a family of lightweight students spanning different encoder types (CNN vs ViT), decoder widths and depths, and training strategies. To bridge the dimensional mismatch between teacher and student, we propose a structured SVD‑based initialization that projects the teacher's decoder weights into the student's smaller latent space, yielding a warm start that significantly improves convergence and final performance. Based on our results on lunar data, we can obtain a distilled student that retains most of teacher's reconstruction accuracy while reducing the model size up to 7 times, and even outperforms a baseline trained directly with sparse ground‑truth annotations. Beyond compression, our study highlights both principles and practical insights for distilling geometric foundation models: a convolutional encoder underperforms transformer‑based alternatives (though pretraining availability remains a confounding factor), preserving encoder capacity is more critical than maintaining a large decoder, feature‑level distillation consistently outperforms output‑only supervision, and SVD‑based initialization improves optimisation stability. These findings provide practical guidelines for deploying 3D reconstruction models in resource‑constrained environments.
Authors:Yuanzhi Liu, Shousheng Zhao, Bo Zhou, Kongming Liang, Zhanyu Ma
Abstract:
Evaluation benchmarks are essential for assessing vision‑language models (VLMs), but most multimodal benchmarks are static, making them vulnerable to temporal staleness, data contamination, and costly maintenance. We present MMBench‑Live, a continuously evolving multimodal benchmark built by a multi‑agent‑driven automated pipeline. Our framework treats benchmark evolution as task‑guided dataset construction, integrating structured benchmark specification, feedback‑controlled real‑time data acquisition, and verifiable QA generation with executable reasoning. To maintain cross‑version comparability, we introduce a distribution‑consistent update strategy that extracts task‑related visual patterns from the original benchmark to guide data collection and filtering. Instantiated from MMBench, MMBench‑Live contains 5.9K newly generated evaluation instances with a high answer correctness rate, while each update costs about USD 30 and takes 1‑2 hours. Extensive evaluations show that MMBench‑Live preserves stable model rankings, maintains semantic alignment with the original benchmark, and exhibits weaker contamination‑related memorization signals, suggesting a practical and scalable paradigm for sustainable multimodal benchmark evolution. The project is available at https://github.com/PRIS‑CV/MMBench‑Live.
Authors:Yunhao Feng, Ruixiao Lin, Ming Wen, Qinqin He, Yanming Guo, Yifan Ding, Yutao Wu, Jialuo Chen, Yunhao Chen, Xiaohu Du, Jianan Ma, Zixing Chen, Zhuoer Xu, Xingjun Ma, Xinhao Deng
Abstract:
LLM agents increasingly perform autonomous actions through external tools, leading to complex and evolving safety risks. However, existing safety testing targets expert‑designed safety violations, and the corresponding outcomes are evaluated by hard‑coded rules, making them costly to extend as agents evolve. To this end, we present Vera, an end‑to‑end automated safety testing framework that instantiates software engineering testing principles for non‑deterministic agents through a three‑stage, self‑reinforcing pipeline. First, a literature‑driven exploration continuously discovers and structures emerging risks into taxonomies of safety risks, attack methods, and tool execution environments. Second, combinatorial composition across taxonomy dimensions produces executable safety cases, each specifying a concrete safety goal, a programmatically constructed initial state, and a deterministic verification predicate grounded in observable artifacts. Third, adaptive execution runs heterogeneous agents in isolated sandboxes where a control agent steers multi‑turn interaction based on runtime observations, while evidence‑grounded verifiers judge outcomes from environment state and tool‑call evidence rather than model self‑report. We evaluate Vera on four production agent frameworks (OpenClaw, Hermes, Codex, Claude Code), revealing substantial safety weaknesses, with average attack success rates reaching 93.9% under multi‑channel attacks; we also release Vera‑Bench, comprising 1600 executable safety cases spanning 124 risk categories across three execution settings. These results indicate that modular, executable testing infrastructure is essential for rigorous and maintainable safety evaluation of rapidly evolving agentic systems at scale. The code is publicly available at https://github.com/Yunhao‑Feng/Vera.
Authors:Jiaxing Wang, Kaitao Chen, Zhubin Han, Chenyu Hou, Bin Cao, Jing Fan, Ji Zhang
Abstract:
Next activity prediction helps service‑oriented processes anticipate upcoming steps before delays, exceptions, or service‑level risks occur. Most existing methods assume classical single‑case event logs, whereas real service processes often involve events shared by multiple typed business objects. Object‑centric event logs (OCELs) capture such interactions, but current predictors remain limited. Flattening‑based approaches lose cross‑object context, and native OCEL graph‑based approaches encode multi‑object events through pairwise relations. Existing models also do not jointly capture event‑driven object state changes, inter‑event timing, and global execution patterns. We propose EHHN, an Event‑driven Heterogeneous Hypergraph Network for object‑centric next activity prediction. EHHN represents each prediction prefix as a heterogeneous hypergraph, where event‑‑object hyperedges bind retained co‑participating objects and a lifecycle hyperedge groups the primary object's observed lifecycle events. Based on this representation, EHHN uses a dual‑stream architecture in which a micro‑spatial stream models event‑driven object‑state evolution and a macro‑evolution stream captures temporal dynamics using retrieved global prototypes. The two streams are fused to predict the next activity. Experiments on four public OCEL benchmarks against nine baselines show that EHHN achieves the best accuracy and macro F1‑score on all datasets, with improvements of up to 8.1 and 12.4 percentage points over the strongest baselines. Compared with the strongest OCEL‑native graph baseline, EHHN also reduces peak GPU memory by up to 24 times. Code is available at https://github.com/chenkaitao1112/EHHN.
Authors:Marianne Arriola, Volodymyr Kuleshov
Abstract:
Discrete diffusion models have steadily improved in quality relative to autoregressive (AR) models. However, these models are normally constrained to fixed‑length generation and do not support key‑value (KV) caching. Block diffusion partially bridges diffusion and AR by generating token blocks left‑to‑right, but its fixed‑size sequential blocks limit decoding flexibility and parallelism. Here, we present a new class of language models, set diffusion, comprised of (i) a likelihood parameterization that factorizes over flexible‑position, flexible‑length token sets and (ii) a set‑causal diffusion architecture that supports KV cache updates after every inference step. By factorizing over token sets instead of fixed‑size blocks, tokens can be decoded in arbitrarily‑ordered sets, including sliding‑window sets, enabling faster inference and support for any‑order decoding. Set diffusion achieves better speed‑quality tradeoffs on mathematical reasoning, summarization, and unconditional generation compared to prior diffusion language models while offering stronger infilling performance than block diffusion. We provide the code, along with the model weights and blog post on the project page: https://m‑arriola.com/setdlms/
Authors:Meng Wang, Haohan Zhao, Wenzhuo Liu, Lu Yang, Geng Liu, Haiyang Guo, Guo-Sen Xie, Gaofeng Meng, Hongbin Liu, Fei Zhu
Abstract:
Continual post‑training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on‑policy learning can mitigate forgetting, with on‑policy self‑distillation emerging as a particularly attractive approach. In this work, we revisit this optimistic view through self‑distillation policy optimization (SDPO). Our experiments show that SDPO can accelerate in‑domain specialization when teacher signals are stable and well aligned, but it struggles to generalize to out‑of‑distribution scenarios. In continual post‑training, SDPO exhibits stronger forgetting and can even collapse, whereas on‑policy reinforcement learning methods such as GRPO adapt more conservatively and better preserve prior capabilities. Further analyses reveal that denser self‑distillation induces larger drift in both parameter space and response space, and can amplify high‑frequency formatting artifacts through a self‑reinforcing teacher‑‑student loop. These findings suggest that on‑policy data alone is insufficient for continual learning. Dense self‑distillation can accelerate specialization when teacher targets are stable and token‑level supervision is reliable, but it should not be treated as a default stabilizer for continual post‑training. Our code is available at https://github.com/Moenupa/SDPO‑CL.
Authors:Shoon Kit Lim, Melissa Jia Ying Chong, Ting Yang Ling
Abstract:
Deep‑learning features excel in visual matching, yet their practical value in tightly coupled visual‑inertial SLAM (VI‑SLAM) remains insufficiently characterized. We present DL‑VINS‑Factory, a unified framework that integrates learned feature extractors (ALIKED, RaCo, SuperPoint, XFeat) with either Lucas‑‑Kanade (LK) optical‑flow tracking or LightGlue (LG) descriptor matching. All front‑ends share a sliding‑window Ceres back‑end, with optional AnyLoc DINOv2‑VLAD loop closure, and 4‑DoF pose‑graph optimization. We benchmark the system across the four datasets covering indoor, unstructured outdoor, aggressive‑motion, and visually degraded conditions. Results show that learned front‑ends are viable for real‑time embedded VI‑SLAM, but are not universally superior to classical tracking. Relative to the corresponding GFTT+LK baseline, ALIKED+LG reduces EuRoC ATE by 5% in monocular odometry and by 7% in stereo with loop‑closure. On NTU‑VIRAL, where aggressive aerial motion increases inter‑frame viewpoint change, ALIKED+LG stereo reduces loop‑closed ATE by 12%. In Botanic Garden dataset, optical‑flow tracking remains preferable, but learned keypoints still improve over the baseline GFTT, in which SuperPoint+LK reduces grayscale camera ATE by 29%, while RaCo+LK reduces RGB camera ATE by 38%. On SubT‑MRS, learned front‑ends display varying degree of improvement based on individual cases. With TensorRT acceleration on a Jetson AGX Orin, all valid configurations run in real time between 29‑‑47 FPS in monocular mode and 18‑‑33 FPS in stereo mode for the EuRoC and NTU‑VIRAL datasets. AnyLoc further confirms roughly 2‑‑7× more valid loops than BRIEF+DBoW2. The implementation is open‑sourced at https://github.com/limshoonkit/DL‑VINS‑Factory‑ROS2/.
Authors:Nikolai Smolyanskiy
Abstract:
We study how to predict the downstream closed‑loop performance of a learned latent world model from validation‑time diagnostics alone. Choosing the right checkpoint from a world‑model training run is difficult: validation loss and multi‑step prediction RMSE keep improving long after closed‑loop performance has collapsed. We present a suite of structural validation‑time diagnostics drawn from optimal‑control theory and apply them to Gymnasium's LunarLander v3, which features shaped rewards. We train an RSSM [5, 4] world model on it and treat per checkpoint CEM‑MPC return as the oracle for closed‑loop quality. By evaluating 40 metrics against this oracle, we find that the strongest single predictor is the Reward Observability Fraction (ROF), which measures the reward predictor's dependence on the observable subspace. We combine ROF with three structural regularizers into a single‑number offline checkpoint selection score, the Composite Reward Observability Fraction (CROF). The CROF‑selected world model trains a model‑based A2C policy that beats a fairly evaluated model‑free A2C baseline by ~24.5 return points while using ~65x fewer real‑environment interactions, and the same world model also drives a strong zero‑shot CEM‑MPC policy. Code and data: https://github.com/nsmoly/LunarLander_RSSM.
Authors:Weiyi Xue, Fan Lu, Chi Zhang, Tianhang Wang, Sanqing Qu, Zehan Zheng, Boyuan Zheng, Junqiao Zhao, Guang Chen
Abstract:
3D Gaussian Splatting has demonstrated remarkable potential in novel view synthesis. In contrast to small‑scale scenes, large‑scale scenes inevitably contain sparsely observed regions with excessively sparse initial points. In this case, supervising Gaussians initialized from low‑frequency sparse points with high‑frequency images often induces uncontrolled densification and redundant primitives, degrading both efficiency and quality. Intuitively, this issue can be mitigated with scheduling strategies, which can be categorized into two paradigms: modulating target signal frequency via densification and modulating sampling frequency via image resolution. However, previous scheduling strategies are primarily hardcoded, failing to perceive the convergence behavior of scene frequency. To address this, we reframe the scene reconstruction problem from the perspective of signal structure recovery and propose SIG, a novel scheduler that synchronizes image supervision with Gaussian frequencies. Specifically, we derive the average sampling frequency and bandwidth of 3D representations, and then regulate the training image resolution and the Gaussian densification process based on scene frequency convergence. Furthermore, we introduce Sphere‑Constrained Gaussians, which leverage the spatial prior of initialized point clouds to control Gaussian optimization. Our framework enables frequency‑consistent, geometry‑aware, and floater‑free training, achieving state‑of‑the‑art performance by a substantial margin in both efficiency and rendering quality in large‑scale scenes. The code is available at: https://github.com/weiyixue999/Signal_Structure_Aware_Gaussian
Authors:Xiong Xiong, Ruonan Zhai, Zheng Zeng, Sheng Zhou, Rongchun Hu, Zichen Deng
Abstract:
Solving partial differential equations (PDEs) with high‑frequency solutions remains a central challenge in physics‑informed machine learning due to spectral bias ‑‑ the tendency of neural networks to learn low‑frequency components preferentially. This paper proposes a Frequency Shift Physics‑Informed Extreme Learning Machine (FS‑PIELM) framework that addresses this limitation through an additive mechanism for weight initialization. Rather than multiplying random weights by a scaling factor, the method translates the mean of the Gaussian weight distribution while keeping the variance fixed at unity, thereby avoiding the variance amplification inherent in scaling‑based methods. Two variants are developed: FS‑PIELM‑L assigns independent frequency magnitudes to individual neurons, while FS‑PIELM‑G groups neurons for improved robustness. Theoretical analysis shows that the frequency variance under the proposed framework remains bounded and approaches unity regardless of target frequency, in contrast to the quadratic growth of conventional approaches. The method preserves the computational efficiency of extreme learning machines, requiring only a single linear solve. Experiments on seven benchmark problems spanning six equation types ‑‑ Helmholtz, wave, Poisson, Klein‑Gordon, heat, and advection‑diffusion ‑‑ on both regular and complex geometries show that the linear variant achieves the best accuracy in six of seven cases, with improvements of one to nearly five orders of magnitude over existing PIELM variants. The code and data accompanying this manuscript will be made publicly available at https://github.com/xgxgnpu/Physics‑informed‑vibe‑coding/tree/main/FS‑PIELM.
Authors:Joshua Penman
Abstract:
Finetuning a language model on documents that are explicitly annotated as fictional results in a model that still actually believes the documents' core claims, an effect known as Negation Neglect. In our evaluations, models trained on documents prefixed and suffixed with such annotations correctly identify the relevant claims as fictional only about 9% of the time. To address this, we introduce Goggles, a learned module that intervenes on the finetuning gradient rather than the data. During supervised finetuning, a Goggles module edits the gradients an LLM LoRA receives, imparting a chosen epistemic frame (the stance the model takes toward the nature of what it reads) to whatever the documents teach. A Goggles instance is trained once for a given base model, frame, and LoRA configuration, then applied frozen to documents it was never trained on. Trained through Goggles on those same documents, now carrying no fictional annotation, the model flags the content as fictional roughly 91% of the time, while preserving capability (GPQA and TruthfulQA match or exceed baseline). The same architecture supports other frames: a Goggles instance can be trained to treat documents as "part of an AI safety evaluation by Redwood Research" rather than simply as fiction. The imparted frame persists under continued finetuning that pushes back toward the claim, where prior interventions revert. Goggles suggests a path toward training language models on known‑misaligned data without absorbing the behaviors that data demonstrates.
Authors:Long Minh Bui, Tuan Anh Le Van, Tung Phi Duc, Phi Le Nguyen, Jana Doppa, Trong Nghia Hoang
Abstract:
Model merging aims to combine existing single‑task solutions into a multi‑task solution without additional data‑driven fine‑tuning.~Most existing approaches achieve this using geometric properties of local solution spaces. However, such geometric views provide limited guidance for scoring how statistically useful each task‑specific update direction is across tasks during merging. To address this, we formulate model merging from a new perspective of probabilistic inference under a product‑of‑experts (PoE) scenario where each single‑task solution defines an energy‑based expert model (EBM) over the merged parameters. We show that several existing model merging methods arise as special cases of our framework under energy designs that impose implicit Gaussian assumptions on directional residuals between merged and task‑specific models. Empirically, we find that these residuals are often heavy‑tailed which exposes a mismatch with the imposed light‑tailed Gaussian structures. We address this with a heavy‑tailed PoE design based on Cauchy experts, which better captures the observed residual behavior while admitting a provably convergent inference procedure. Experiments across multiple tasks and architectures show significant improvements over state‑of‑the‑arts baselines. Our code is available at https://github.com/MinhLong210/PoE‑EBM‑Merging.git.
Authors:Xuanhua He, Jiaxin Xie, Mingzhe Zheng, Qifeng Chen
Abstract:
Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per‑frame accuracy but suffer from temporal drift due to limited context windows, while generative methods improve consistency and generalization at the cost of extensive training data (10M+ samples) and lack of geometric precision. In response to these issues, we introduce ICDepth, a framework that adapts pre‑trained text‑to‑video diffusion transformers for video depth estimation via In‑Context Conditioning (ICC), leveraging their rich spatial‑temporal priors. To address key challenges in transferring ICC from generation to dense prediction, we propose: (1)~SAND‑Attention, which ensures precise spatial‑temporal alignment via shared RoPE and enforces unidirectional attention to prevent noise contamination; (2)~SRFM, which injects DINOv2 semantic and resolution priors to enhance geometric precision. ICDepth achieves state‑of‑the‑art results on multiple benchmarks with remarkable data efficiency, trained on only 0.8M frames (6‑‑13× less than competing generative methods), while demonstrating strong zero‑shot generalization to diverse domains.
Authors:Saad Wazir, Rao Faizan, Daeyoung Kim
Abstract:
Segmentation of biomarkers in medical images is frequently viewed as a first step towards medical image analysis in any bioinformatics or biomedical application. Despite progress, existing methods still struggle to capture information at multiple scales and to perform upsampling effectively across different datasets. These shortcomings often result in suboptimal generalization capabilities. Recently, architectures belonging to the Nested‑UNet family excel in capturing multiscale contextual information and upsample them effectively. In this work, We propose a novel Nested‑UNet architecture that effectively captures multi‑scale contextual information. It includes inner and outer attention units to enhance focus during upsampling, along with channel‑wise feature recalibration using squeeze‑and‑excitation modules, leading to improved segmentation performance. Additionally, the architecture integrates an edge‑aware loss to emphasize boundary accuracy by assigning greater importance to edge regions. Tested extensively on three publicly available benchmark datasets. Our method demonstrates a generalization performance superior to existing Nested‑UNet methods. Code: https://github.com/saadwazir/histosegplusplus
Authors:Yuguang Yang, Canyu Chen, Zhewen Tan, Yizhi Wang, Zichao Feng, Chunyang Liu, Kehua Sheng, Juan Zhang, Linlin Yang, Baochang Zhang, Yan Wang, Bo Zhang, Xianbin Cao
Abstract:
Vision‑Language‑Action (VLA) models have emerged as a promising paradigm for end‑to‑end autonomous driving. However, existing VLAs' training relies heavily on text‑centric visual question answering and chain‑of‑thought reasoning data, which emphasizes linguistic reasoning rather than action‑grounded planning. As a result, the learned representations capture semantic knowledge but lack spatial dependencies crucial for reliable trajectory prediction. We propose DriveTeach‑VLA, a framework that explicitly teaches VLAs what to see and where to look. Driving‑aware Vision Distillation (DVD) injects driving‑specific perceptual priors into the vision encoder, while 2D Trajectory‑Guided Prompts (2D‑TGP) provide spatial conditioning aligned with feasible driving trajectories. Together, they form a vision‑guided learning pipeline: what to see (DVD pretraining) ‑ where to look (TGP‑guided SFT) ‑ how to act (TGP‑guided GRPO). DriveTeach‑VLA achieves the state‑of‑the‑art performance on NAVSIM and nuScenes. Our code is available at: https://github.com/ShivaTeam/DriveTeach‑VLA.
Authors:Yuwan Liu, Hongze Yu, Song Liu, Yuhan Wang, Junge Zhang, Yaodong Yang, Yuanpei Chen, Ceyao Zhang
Abstract:
Learning effective robot control policies on physical hardware is challenging due to costly data collection and the difficulty of reward specification. Prior work has incorporated demonstrations into reinforcement learning (RL), yet existing approaches either require large numbers of demonstrations or depend on continuous human intervention during training. To address these limitations, we present AutoSERL, a framework that leverages a single demonstration to fully automate the intervention process in real‑world robot RL. The framework includes three complementary mechanisms to accomplish certain tasks: a sliding window intervention mechanism that continuously guides exploration to prevent local optima and unsafe deviations, a safety recovery mechanism that detects and corrects failure states via predefined trajectory recovery points, and an intervention termination criterion that automatically disables guidance once the policy can independently complete the task, preserving its exploration advantage. We evaluate AutoSERL on six contact‑intensive manipulation tasks across two robot platforms, spanning insertion, hanging, and hinge‑based tasks. AutoSERL consistently outperforms SERL initialized with 20 demonstrations, behavior cloning, and MILES ‑‑ a dedicated one‑shot imitation learning baseline ‑‑ across all tasks while matching HIL‑SERL, achieves 100% success rate on insertion tasks, and demonstrates improved robustness to positional variations, all from a single demonstration. Code and videos are available on our project website: https://autoserl.github.io/.
Authors:Bo Zhao, Yapeng Li, Juhua Liu, Bo Du
Abstract:
Ultrasound image classification is essential for computer‑aided diagnosis. However, current methods often neglect clinical priors, leading to poor generalization in challenging scenarios and a lack of interpretability that limits clinical adoption. To address these issues, we aim to develop a medical‑prior module that can be seamlessly integrated into existing pipelines to enhance both diagnostic performance and interpretability. In this paper, we propose an attribute‑guided dual‑branch framework for ultrasound classification that introduces domain‑agnostic medical attribute priors, improving generalization while offering interpretable evidence. Specifically, a baseline branch follows conventional architectures and predicts image categories via a fully connected classifier. An attribute‑guided branch injects domain‑agnostic attributes as priors and produces human‑interpretable decision cues. Finally, an adaptive decision module fuses the two branches in a data‑dependent manner to yield the final prediction. Experiments across diverse ultrasound classification tasks demonstrate that our approach can be integrated into multiple backbones and state‑of‑the‑art methods with low overhead, consistently improving accuracy and interpretability. Code is available at: https://github.com/zhaobo253‑crypto/AttrGuide.
Authors:Xingyu Zheng, Xianglong Liu, Yifu Ding, Weilun Feng, Junqing Lin, Jinyang Guo, Haotong Qin
Abstract:
Hardware‑agnostic strategies for accelerating text‑to‑image diffusion, such as timestep distillation and feature caching, can reduce inference time without custom kernels or system‑level optimization. Among them, multi‑resolution generation strategies have recently received broad attention, attaining more than 5x speedup without any training. However, the design of performing upsampling in the latent space, together with the selective modification of partial regions, causes these methods to exhibit noticeable blurring or artifacts. To this end, we propose MrFlow, a training‑free multi‑resolution acceleration strategy for pretrained flow‑matching models built upon a staged low‑to‑high‑resolution pipeline. MrFlow first rapidly generates the main structure at low resolution, then performs super‑resolution in the pixel space using a lightweight pretrained GAN‑based model, subsequently injects low‑strength noise to enable high‑frequency resampling, and finally refines the details at high resolution. Quantitative and qualitative results on FLUX.1‑dev and Qwen‑Image show that MrFlow exploits the quadratic token reduction and reduced step requirement of low‑resolution sampling to achieve 10x end‑to‑end acceleration while keeping OneIG within a 1% gap relative to that before acceleration, significantly surpassing other training‑free acceleration strategies, and requiring no training or runtime dynamic identification whatsoever. MrFlow can further be directly combined orthogonally with pre‑trained timestep distillation strategies, achieving even higher generation acceleration of up to 25x.
Authors:Jeongwan On, Muhammad Salman Ali, Muneeb A. Khan, Sunwoo Park, Inwoong Moon, Hyung Jin Chang, Jaekwang Kim, Seong Jong Ha, Seungryul Baek
Abstract:
Tracking multi‑person 3D human meshes from in‑the‑wild videos is a highly challenging problem due to complex interactions, frequent occlusions, and severe truncation inherent in unconstrained environments. While recent approaches have improved robustness against these issues, they largely overlook the critical challenge prevalent in real‑world footage: frequent shot changes. These abrupt transitions in camera viewpoints often cause existing methods to lose track of human identities and fail in reconstructing temporally coherent trajectories. Although several recent works have explored 3D human mesh tracking under shot changes, they are still limited to single‑person scenarios, making them inadequate for real‑world videos where multiple people interact and appear simultaneously. To address this limitation, we propose Multi‑THuMBS (Multi‑person Tracking of 3D Human Meshes Beyond Video Shots) that leverages a state‑of‑the‑art 3D scene prior to reconstruct the two boundary frames in a single shared 3D space. Human meshes are then registered within the shared 3D space, maintaining per‑person identity and motion consistency across shot changes. Extensive experiments demonstrate that our approach yields significant improvements in 3D human mesh recovery, camera pose estimation, and identity tracking, thereby ensuring high‑fidelity motion reconstruction with consistent identity preservation across shots compared to previous state‑of‑the‑art methods.
Authors:Yujie Guo, Jiaming Zhou, Yuhang Jia, Yang chen, Yong Qin
Abstract:
Multi‑talker Automatic Speech Recognition (MTASR) faces significant challenges in accurately transcribing overlapping speech, particularly under complex high‑overlap conditions. While recent Mixture‑of‑Experts (MoE) approaches have shown promise, they typically rely on frame‑independent routing that leads to temporal myopia, and depend solely on the downstream ASR objective, which results in implicit and ungrounded representation learning. To address these limitations, we propose Holistic Speaker‑Aware Guided Experts (H‑SAGE) for MoE‑based MTASR. Specifically, we introduce a Speaker‑Aware Global Encoder to capture long‑term dependencies, supervised by an auxiliary Overlap‑Aware Loss that explicitly guides the model to discern acoustic states. Furthermore, we design a Holistic Gating Mechanism to arbitrate expert selection by jointly evaluating global context and local details. Experiments on LibriSpeechMix demonstrate that H‑SAGE achieves consistent improvements over strong baselines, particularly in complex scenarios, validating that explicit acoustic guidance effectively enhances expert collaboration. Our code can be found at https://github.com/NKU‑HLT/H‑SAGE.
Authors:Leyan Li, Rennong Yang, Zhenxing Zhang, Liping Hu
Abstract:
Transformers have become general‑purpose architectures, but their all‑to‑all self‑attention is poorly matched to graph data, whose interactions are sparse, structured and multi‑scale. Existing Graph Transformers address this mismatch through structural encodings, hybrid message‑passing modules or learned attention constraints, often introducing additional complexity and limited interpretability. Here we introduce X‑LogSMask, an explainable multi‑head logarithmic structural mask that injects symmetrically normalized graph topology directly into attention logits. The logarithmic transform converts structural connectivity into a topology‑aware gating signal, suppressing unsupported node interactions while preserving feature‑dependent attention. By assigning different powers of the normalized adjacency matrix to different attention heads, X‑LogSMask gives each head a defined structural radius and supports multi‑hop information propagation within a single layer. We further show that a standard Transformer encoder can be interpreted as one‑step message passing on a complete graph, motivating X‑LogSMask as a topology‑constrained alternative to unrestricted self‑attention. Across 20 node‑, edge‑ and graph‑level benchmarks, Transformers equipped with X‑LogSMask achieve state‑of‑the‑art performance on 13 datasets and remain competitive in a lightweight one‑layer configuration. These results show that simple, interpretable structural masks can make self‑attention an effective graph‑learning operator without changing the Transformer architecture. The code is available at https://github.com/LiLeyan‑0120/X‑LogSMask.
Authors:Natalie Grace Brigham, Eugene Bagdasarian, Tadayoshi Kohno, Franziska Roesner
Abstract:
AI agents that autonomously execute tool calls on a user's behalf raise pressing questions about permission management: what role could users play, and what role should they play? Despite many proposed approaches, the user's role in agentic permission management remains under explored. We introduce Janus, a playground system for implementing and evaluating user‑involved agentic permission management designs. Janus consists of two components: Janus‑Core, a modular agentic system supporting a diverse spectrum of permission management designs, and Janus‑Harness, an automated evaluation framework. Grounded in a conceptual model that identifies key design axes for user involvement, we implement six permission assistants spanning the design space and evaluate them across three scenarios and three synthetic responders. We demonstrate that user input is critical and can significantly strengthen privacy and security, that AI augmentation of user decisions can help reduce cognitive load, and that realistic user behavior including permission fatigue must be accounted for in system design. No single design performs optimally across all contexts, motivating a more principled and context‑sensitive approach to deploying permission assistants in agentic systems. Janus is publicly available to support future investigation into this dimension of agentic system design.
Authors:Yiqian Liu, Iuliia Kotseruba, John K. Tsotsos
Abstract:
In this paper, we study depth perception of vision‑language models (VLMs) to isolate the effects of pictorial depth cues and disentangle vision and language influences on model performance. To this end, we combine depth‑ordering and odd‑one‑out psychophysical tasks: the VLMs are presented with images where one object is at different depth relative to other, otherwise identical, objects, and must determine whether the odd‑one‑out target is closer or farther to the observer. To create stimuli, we generate 2D views from simulated and real 3D scenes while controlling the presence of individual pictorial depth cues, enabling a fine‑grained analysis of cue‑level contributions. Language effects are examined by varying referring expression clarity. We also introduce a novel metric to quantify vision‑vs‑language sensitivities. Applying this methodology, we create the Odd‑One‑Out Depth (O3‑D) dataset with 37K real and synthetic images and 147K image‑question pairs. Evaluation of 12 open‑source and commercial models on O3‑D shows under‑utilization of depth cues and depth‑ordering accuracies between 47% and 56%, with no model above chance level. At the same time, our metric reveals strong linguistic bias in the answers. Neither chain‑of‑thought (CoT) nor in‑context learning (ICL) significantly improves performance, suggesting that static image data alone may be insufficient for depth understanding. All code, the image generation pipeline, and the O3‑D dataset are publicly released at https://github.com/lyiqian/o3‑d.
Authors:Kevin Wang, Kevin Yang, Arjun Prakash, Amy Greenwald
Abstract:
We investigate the problem of learning useful policy representations (embeddings) in two‑player zero‑sum imperfect‑information games. We make three contributions: First, we introduce methods of creating datasets of policies for a given game. Second, we propose methods to learn policy representations. Third, we introduce downstream tasks to evaluate the effectiveness of such representations. We evaluate each dataset method, embedding method, and downstream task on Kuhn and Leduc Poker. Although our methods are very basic, we demonstrate that useful behavioral representations are present in the learned embeddings. To our knowledge, this work is among the first to systematically compare self‑supervised learning techniques for learning policy representations in games. Our code is available at https://github.com/VitamintK/ssl‑project for others to extend.
Authors:Aseel Mohamed, Rama AlHamidi, Mohamed Rayan Barhdadi, Rasul Khanbayov, Erchin Serpedin, Hasan Kurban
Abstract:
Agentic Video Question Answering (VideoQA) systems invoke tools during inference, but their tool libraries are fixed, so recurring procedures are rebuilt from primitives on every question. Synthesizing composite tools could remove this overhead, but whether such expansion helps is hard to assess: final‑answer accuracy, the standard metric, ignores inference effort, so it cannot reveal how a system shifts cost. We propose a cost‑aware, paired protocol for auditing tool‑augmented video agents. The protocol pairs two complete systems on the same input for each question and reports their net difference across accuracy and cost jointly. For each question, it sorts the paired outcome into one of six groups defined by joint correctness and by the change in visible tool calls, separating accuracy‑preserving efficiency gains from harmful regressions. Significance is reported with McNemar's test and paired bootstrap confidence intervals. We instantiate the protocol on Dynamic‑SAGE, an agentic VideoQA framework that synthesizes, validates, and persistently registers executable composite tools for reuse on unseen questions, and evaluate it against the SAGE baseline on SAGE‑Bench. The audit reveals a multi‑axis profile that a scalar accuracy comparison would miss: Dynamic‑SAGE improves accuracy by 7.5 points (p < 0.001) and reduces reasoning turns and visible tool calls by roughly 28%, while shifting rather than reducing inference cost, as token usage rises 34% and cost 26%. Gains are largest on visual and open‑ended questions and neutral on verbal and multimodal ones, and residual failures concentrate on hard, open‑ended questions where the pipeline does the most work. By measuring accuracy and cost jointly, the protocol shows where the pipeline‑level difference is reliable and where it is not. The code is available at https://github.com/KurbanIntelligenceLab/Dynamic‑SAGE.
Authors:Shashank Indukuri, Adarsh Agrawal
Abstract:
Large language models (LLMs) are increasingly applied to resume optimization for applicant tracking systems, introducing hallucination failures distinct from general text generation: anachronistic technology injection, cross‑domain terminology contamination, structural mutation, and content fabrication. We present Grounded Optimization, a five‑layer framework combining temporal context validation, deterministic contamination detection, structural invariant enforcement, prompt‑level grounding, and an evaluator agent. In ablation experiments across three LLMs, four temperature settings, and six layer configurations on 25 synthetic resumes spanning 14 industries, undefended baselines produce 2.48‑5.36 detected hallucinations per resume. Among detectors independent of the active defenses, temporal hallucinations are reduced by 50‑95% across all conditions; overall detected hallucination rate falls to 0.04‑0.24. Prompt‑level grounding alone achieves zero detected hallucinations at low temperature with a capable instruction‑following model; higher temperatures and weaker models reveal the need for the deterministic layers as a complement. We release the contamination taxonomy, evaluation code, and raw data.
Authors:Kathan Shah
Abstract:
Language models learn continuous programs over discrete symbols, with the embedding table and LM‑head acting as the read/write interface between them. We show that this interface has gradient geometry distinct from dense hidden weights which can be exploited to improve the Pareto frontier across supervised finetuning, RL, and pretraining, while only utilizing kilobytes of optimizer state. We introduce Ember, a lightweight optimizer for embedding and LM‑head matrices that utilizes O(V + D) VRAM, instead of Adam's O(2VD), and forgoes the need to shard both token table optimizer states. We provide empirical evidence that Ember scales effectively across batch size and parameter count. We show that the optimization trajectory of tokens can be well described by a simple 1D ray, counter to the popular belief that neural net parameters navigate a heavily nonconvex landscape. We provide a principled view on the surprisingly narrow space of optimizers that suffice for Transformer training. Finally, we open‑source our distributed Ember implementation that merges cleanly with existing ZeRO/FSDP setups to support further research at https://github.com/katop1234/ember
Authors:Foad Namjoo, Drew McClelland, Michael Matheny, Jeff M. Phillips
Abstract:
Anomaly detection in geospatial data is a crucial tool in geographic information science (GIS), with applications ranging from national security to public‑health surveillance to the study of societal disparities. This work focuses on spatial scan statistics and addresses a key mismatch: spatial counts are typically aggregated into predefined regions (census tracts, zip codes, counties), whereas the most efficient scan algorithms operate on spatial point data. The standard remedy ‑‑ collapsing each region to its centroid, as in widely used tools such as SaTScan ‑‑ is convenient but, as we show, discards the region's spatial extent and causes a significant loss in statistical power. To resolve this, we propose a simple yet scalable fix: replace each spatial region with 20‑50 points sampled uniformly from its geometry and spread the region's values evenly across them. This approach improves statistical power while maintaining computational tractability. A convergence analysis explains why so few samples per region suffice. We recommend this sampling‑based conversion as the default way to apply point‑based spatial scan statistics to region‑aggregated data for anomaly detection.
Authors:Zhiyun Zhang, Liwen Sun, Xiang Qian, Chenyan Xiong
Abstract:
Faithful reasoning is essential in medicine, where clinical decisions require transparent justification grounded in reliable evidence. Current medical LLMs either lack active access to evidence or use retrieved evidence without supervising how it should be appraised and applied during reasoning. To address this, we formalize evidence‑based medicine principles as process‑level criteria and introduce FaithMed, a framework that combines clinician‑designed, automatically refined rubrics with reinforcement learning using step‑level process reward assignment and advantage grouping. Across seven medical benchmarks, FaithMed improves over agentic‑search baselines (+9% on average) and outcome‑only RL (+5.8%), while raising average evidence‑based medicine rubric scores over agentic‑search Qwen3 baselines (+15.5%). This work demonstrates that explicit step‑level supervision can improve both task success and the faithfulness of the reasoning process. Code is available at https://github.com/cxcscmu/FaithMed.
Authors:Parv Agarwal, Asif Ekbal
Abstract:
GPU training jobs fail often, roughly two in five on large production clusters, yet the operator typically learns of a failure only by reconnecting hours later. Experiment trackers require editing the training script and maintaining a cloud connection; the scheduler's mail hook delivers a single status line with no cause and no logs. GPUAlert is a command‑line wrapper that monitors any training command at the process boundary, and with no change to that command, emails a structured notification on completion carrying a classified failure cause, durable logs, and output artifacts. The tool is organized around three reliability primitives: a pre‑launch log guarantee that establishes the durable destination before the child process can crash, notifier isolation that makes the wrapper's exit code a pure function of the child's status regardless of whether the email succeeds, and a non‑silent artifact budget that bounds attachment size without ever dropping output silently. We release a labelled corpus of 474 GPU training logs across 15 failure classes and a reproducible evaluation harness. On the twelve hardware‑reproduced classes, the ordered‑rule classifier reaches 0.997 macro‑F1, against 0.830 for unordered keyword matching and 0.133 for exit‑code inspection. Wrapper overhead is a constant approximately 3ms per job; the pre‑launch guarantee preserves a log where a shell redirect yields nothing; and across all 15 failure modes the wrapper returns the child's exit code unchanged even when the SMTP relay is unreachable.
Authors:Barada Sahu, Shivesh Pandey
Abstract:
Deep multimodal brain‑encoding models now predict fMRI responses to naturalistic video with high accuracy. Whether their predicted neural signals also forecast behavioral engagement is unknown. We run TRIBE, the winning model of the 2025 Algonauts brain‑encoding challenge (Llama‑3.2 + V‑JEPA2 + Wav2Vec‑BERT), on 48 YouTube videos and reduce its predicted cortical response to a per‑second engagement curve, the global field power. Correlated against each video's "most replayed" heatmap, a passively‑collected proxy for which moments viewers return to, the curve shows no evidence of predicting re‑watch behavior. The pooled position‑controlled partial correlation is +0.058 (95% CI [‑0.04, 0.15]; one‑sample t(47)=1.21, p=0.23), indistinguishable from zero and not significantly above simple loudness and motion baselines (loudness +0.04, paired p=0.74). The raw correlation is also near zero; the moderate values reported for music videos reflect a genre‑specific intro/onset‑replay artifact rather than content prediction, and do not generalize. The null holds across six cortical‑network readouts and under an autocorrelation‑preserving permutation test. We release the code, the video‑ID manifest, and an acquisition method that works despite YouTube's SABR‑only streaming.
Authors:Yunfei Bi, Youran Wang
Abstract:
Motion planning algorithms should be evaluated in human‑in‑the‑loop environments to ensure they produce safe and efficient behaviors during interactions. However, existing simulation platforms often rely on recorded datasets, lack dedicated interfaces for real‑time human interaction, or remain weakly integrated with an autonomous driving ecosystem. Moreover, many human‑in‑the‑loop simulators are computationally intensive by design, making them less suitable for rapid prototyping and flexible experimentation in early‑stage autonomous driving research. To address these limitations, we present CommonRoad‑Game, a lightweight human‑in‑the‑loop simulation framework tightly integrated with the CommonRoad platform, focusing on the systematic testing of motion planners with human participation and the analysis of human driving behaviors in interactive scenarios. We introduce a multi‑threaded architecture with a robust synchronization mechanism that aligns simulation time with wall‑clock time, enabling deterministic and temporally consistent interaction between autonomous and human‑driven vehicles. In addition, the framework provides a scenario generation module that records driving logs, allowing diverse and reproducible test cases to be constructed from human‑in‑the‑loop experiments. Experimental results demonstrate that CommonRoad‑Game achieves stable temporal synchronization, supports scalable multi‑agent simulation, and seamlessly integrates CommonRoad‑compatible motion planners to generate interactive driving scenarios. The source code is publicly available at https://github.com/Yunfei‑Bi8/CommonRoad‑Game.
Authors:Yanxiong Li, Jiaxin Tan, Qianqian Li, Guoqing Chen, Sen Huang, Tuomas Virtanen
Abstract:
Most existing audio classification methods suppose that each query (testing) sample belongs to a class of support (training) samples, and misrecognize samples of unseen classes as seen classes (cannot reject samples of unseen classes). In this study, we propose a method for Few‑shot Open‑set Audio Classification (FOAC), which can recognize query samples of seen classes after updating the model using a few support samples, and meanwhile reject query samples from unseen classes. We design a model consisting of an encoder and a classifier. The encoder is the backbone of a ResNet used for extracting embeddings. The classifier consists of prototype generators of few‑shot classes and open‑set classes. Prototypes of few‑shot classes are obtained by fusing the class‑discriminative information of support and query embeddings and by assigning larger weighting coefficient to representative part of the support embeddings. One prototype is generated for open‑set classes using the proposed prototype generator. The encoder is trained with abundant samples of base classes in supervised manner, and then the prototypes of base classes are generated under the supervision of a joint loss. The classifier is trained using a few samples of few‑shot classes in a meta‑training way. Three public datasets (LS‑100, NSynth‑100, and FSC‑89) are used to assess the performance of our method. Experiments show that our method has advantage over prior methods in AUROC and accuracy. This advantage has statistical significance for most prior methods. Our method has lower computational complexity than most prior methods. The code is at https://github.com/Jessytan/FOAC‑AIFP.
Authors:Matthew J Liu, Wei Hang Zheng, Vidhan Purohit, Siqi Xie, Chieh-En Li, Jerry Li, Noah Flynn
Abstract:
Grid‑based approaches to approximate nearest neighbor (ANN) search have been absent from modern scaling analyses. We present a systematic characterization of a multiprobe grid algorithm with respect to dataset size N and dimensionality d. Our experiments reveal a previously unreported d‑scaling crossover on the GloVe embedding family, in which multiprobe grid search maintains an approximately constant dimensional scaling exponent while other graph‑, tree‑, and partitioning‑based methods exhibit degrading throughput. The advantage comes with near‑linear query scaling in N, but also with lower indexing cost than competing ANN methods. Our results suggest that grid‑based methods such as multiprobe grid may be competitive in rebuild‑heavy or high‑dimensional settings where indexing cost and dimensional robustness dictate performance. More broadly, recent work has formalized self‑attention as an ANN operation. Thus, the N‑ and d‑scaling properties of ANN algorithms may guide cost analysis of efficient transformer architectures. Code is available at: https://github.com/weiz345/MultiProbeANN.
Authors:Yuyang Jiang, Chacha Chen, Teng Wu, Liwen Sun, Han Liu, Shi Feng, Chenhao Tan
Abstract:
Debate, where AI agents argue opposing positions, has emerged as a key approach to scalable oversight. However, debate faces a fundamental tension: models are incentivized to be persuasive to the judge, which may not always align with epistemic honesty. In this work, we propose an alternative paradigm: disagreement resolution, which reframes the interaction mechanism from adversarial debate to collaborative truth seeking. Drawing on principles from human mediation and conflict resolution, where mediators facilitate dialogue to help disputing parties reach consensus rather than adjudicating between them, we design an automated pipeline that adapts these strategies to AI oversight. Unlike standard debate where models argue for fixed positions, our pipeline directs models to collaboratively identify points of disagreement, examine the evidence for conflicting claims, and converge toward consensus or isolate the specific ''crux'' of their disagreement. We find that Disagreement Resolution consistently helps non‑expert models identify the truth, achieving 62.1% judging accuracy compared to 49.2% for standard debate. Our results provide encouraging empirical evidence for rethinking the scalable oversight protocol from adversarial persuasion to collaborative truth‑seeking.
Authors:Yue Han, Chong Li, Zhening Liu, Cong Huang, Fang Deng, Yong Liu, Fangyun Wei, Yan Lu
Abstract:
Recent 3D generative models can synthesize high‑quality geometry but often struggle to reproduce intricate textures from reference images, largely due to the scarcity of large‑scale 3D training data with rich surface appearance. In contrast, visual generative models are trained on datasets several orders of magnitude larger and excel at modeling complex visual patterns. Motivated by this gap, we introduce Ink3D, a framework that bridges 3D generation with large‑scale video generative models to synthesize extremely complex textures. Ink3D first reconstructs a white‑mesh geometry using an off‑the‑shelf 3D generation model. It then employs OrbitPainter, a conditional video generative model, to produce dense orbit‑scan videos capturing object appearance across viewpoints. To convert these views into coherent textures, we introduce TextureOptimizer, a neural baking module that integrates dense multi‑view observations while mitigating geometry inconsistencies arising from video generation. By decoupling geometry and texture synthesis and leveraging large‑scale pretrained video priors, Ink3D enables significantly richer and more faithful texture generation than prior approaches.
Authors:Chenyang Ma, Yue Yang, Radu Corcodel, Siddarth Jain, Andrew Wu, Chiori Hori, Diego Romeres
Abstract:
Current work on robot furniture assembly mostly focuses on toy‑scale settings or single‑arm manipulation. We introduce FurnitureVLA, the first systematic study of real‑scale bimanual furniture assembly using Vision‑Language‑Action models (VLAs). We formalize the task, develop a scalable simulation pipeline for expert data generation and evaluation, and build a VR teleoperation system for single‑operator bimanual control to collect high‑quality real‑world demonstrations. To address extreme long‑horizon assembly with up to 7 subtasks and 1550 control steps, we propose a progress‑enhanced VLA, finetuned on semantically grounded subtasks, that jointly predicts actions and a continuous progress signal, enabling automatic subtask transitions and reducing compounding errors during inference. We further study perception and control design factors that critically affect precision in real‑scale assembly. FurnitureVLA improves average simulation success from 48% to 80% compared to baselines across three furniture types, with an additional 21% gain from our design factor study. We validate on a real Kinova Gen3 platform with only 16% drop on the hardest task.
Authors:Anushrut Jignasu, Daniele Grandi
Abstract:
We present Linkify, a framework for learning from interface‑augmented assembly graphs to enable context‑aware part retrieval in mechanical assemblies. While recent generative AI methods for CAD have focused largely on isolated parts or monolithic assemblies, the rich geometric information at the interfaces between parts, where function is realized, remains underexplored. We address this gap by recomputing high‑fidelity interface geometry for the Fusion 360 Gallery Assembly dataset, correcting missing and erroneous contacts, and generating point‑cloud representations of local contact regions. Using this data, we construct assembly graphs whose nodes encode part geometry and whose edges encode interface geometry via a pretrained point‑cloud encoder. On top of this representation, we train a Graph Attention Network based on GATv2 to solve a masked part prediction task: given an assembly with one part held out, the model predicts the class of the missing component from a large vocabulary of geometrically clustered parts, thereby approximating a realistic part‑retrieval scenario. Compared to non‑graph baselines such as logistic regression and k‑nearest neighbors operating on aggregated node features, Linkify achieves higher Top‑K accuracy and F1 scores. Ablation studies on graph connectivity, edge attributes, and attention mechanisms demonstrate that accurate contact computation and dynamic attention over interfaces are critical for performance. Our corrected interface dataset and training pipeline, released publicly, provide a foundation for future interface‑aware models for assembly retrieval, validation, and generative design.
Authors:Chuanming Yu, Jiaming Liu, Zihao Ge, Xiongfei Wu, Lulu Zhu, Pengzhan Zhao, Jianjun Zhao
Abstract:
Quantum computing has emerged as a promising computational paradigm for machine learning (ML), with the potential to offer computational advantages over classical approaches. At this stage, the evidence supporting the performance and advantages of quantum machine learning (QML) models relative to classical models is insufficient. To address this gap, this paper presents an empirical study on the performance of QML models and their classical counterparts. We compare seven model pairs spanning supervised learning and reinforcement learning. Our results indicate that the evaluated quantum machine learning models do not yet surpass the classical baselines in overall prediction performance, policy stability, or training time. Nevertheless, QML remains a promising approach for filtering noise and controlling false positives. Our research findings summarize the challenges facing quantum machine learning across hardware environments, training efficiency, and convergence stability, providing a foundation for research into the robustness and parameter optimization of QML. This work is publicly available at https://github.com/Z‑537‑437/QML.
Authors:Hongxing Li, Xiufeng Huang, Dingming Li, Wenjing Jiang, Zixuan Wang, Haolei Xu, Hanrong Zhang, Haiwen Hong, Longtao Huang, Hui Xue, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Abstract:
Fine‑grained visual reasoning remains challenging for vision‑language models, especially when small but critical visual cues are buried in high‑resolution images. Existing approaches rely on repeated cropping or test‑time visual search to introduce local evidence, but they typically do not explicitly distinguish perception from reasoning. In this paper, we propose Perceive‑to‑Reason (P2R), a unified framework that formulates fine‑grained visual reasoning as a two‑stage process: the model first localizes question‑relevant evidence as a Perceiver, and then answers the question as a Reasoner based on the annotated image and cropped regions. To better align training with this decoupled formulation, we further introduce Perception‑Reasoning Alternating GRPO (PRA‑GRPO), a role‑aware reinforcement learning strategy that alternates between perception‑focused and reasoning‑focused updates using only final‑answer supervision. Built on top of Qwen3‑VL‑Instruct‑2B/4B/8B, P2R consistently improves performance across model scales. In particular, P2R‑4B achieves 93.2% on V‑Star, 81.9% on HR‑Bench‑4K, and 80.5% on HR‑Bench‑8K, substantially outperforming its corresponding backbone. Further experiments show that the benefits of P2R extend beyond high‑resolution benchmarks to broader multimodal reasoning tasks. These results suggest that explicitly decoupling perception from reasoning provides an effective framework for fine‑grained visual reasoning.
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Filter‑and‑refine spatial joins have always avoided touching exact geometry for certified candidate pairs, but the field never modeled the decompression cost of the pairs that survive the filter. When geometry is stored in a compressed, progressively‑decodable multiresolution codec, the join's true cost is bytes decoded. We study provably‑exact polygon intersection joins over a Douglas‑Peucker level‑of‑detail (LOD) ladder, certified by a two‑sided Hausdorff‑margin test, and make two contributions. First, a reproducible mechanism and harness: on real U.S. Census TIGER water polygons, our progressive certificate join returns the exact join result while decoding 3.4‑16.8x (median 5.9x) fewer vertices than naive decompress‑then‑refine, and about 4.9x fewer than the single‑approximation multi‑step baseline of Brinkhoff et al. (1994), with zero correctness violations (set‑equality against a full‑precision oracle) across 31 workloads. Second, a characterization we call the decode‑work law: decode work is governed by each pair's signed‑clearance margin ‑‑ how close it is to the predicate‑flip boundary ‑‑ independent of object size, because the certificate descends the ladder only until its resolution beats the margin. The law is clean on controlled geometry (held‑out R2=0.87, size‑independent) and directional on real data (R2 ~= 0.55). We are explicit about what does not hold: a near‑boundary‑vertex predictor is the wrong model (we pre‑registered one and rejected it), a selectivity regime forecaster did not materialize, and the worst case is the trivial Omega(v) read bound on adversarially interleaved boundaries. We contribute the mechanism, budget‑honest decode accounting, and an open harness; we do not claim a new index.
Authors:Yu Guan, Tianjia Huang, Qinrong Cai, Qiuyun Fan, Dong Liang, Qiegen Liu
Abstract:
Magnetic resonance imaging (MRI) reconstruction under realistic acquisition conditions can be fundamentally viewed as estimating the underlying k‑space distribution from incomplete and noise‑corrupted measurements. While diffusion models have recently shown strong potential as generative prior for inverse problems,existingapproachesstruggletohandlenoisyreconstruction settings, especially when operating directly in k‑space domain. In this work, we propose a unified high‑dimensional k‑space reconstruction framework tailored for noisy inverse problems, whichenhancesdiffusion‑based solversthroughrepresentation lifting.Ratherthanmodifyingthe underlying optimization procedures, the proposed framework augments the data representation space, enabling existing diffusion‑based solvers to operate on enriched k‑space embeddings with improved expressiveness. Extensive experiments on both in‑house and public datasets across varying noise levels and undersampled factors demonstrate that the proposed frame work consistently improves reconstruction quality for multiple diffusion‑based inverse solvers. Notably, the largest gains are observed in high‑noise regimes, which is consistent with our theoretical analysis of error propagation under high‑dimensional representation. These results suggest that high‑dimensional representation provides a general and model‑agnostic mechanism for improving diffusion‑based MRI reconstruction in noisy settings, offering a new perspective on robust k‑space generative modeling for practical inverse problems. The code will be available at https://github.com/yqx7150/HEP‑MRIRec.
Authors:Zhiyi Li, Peilin Wu, Xiaoshen Han, Ruojin Cai, Yilun Du
Abstract:
Video predictive models are emerging as a powerful paradigm in robotics, offering a promising path toward task generalization, long‑horizon planning, and flexible decision‑making. However, prevailing approaches often operate on 2D video sequences, inherently lacking the 3D geometric understanding necessary for precise spatial reasoning and physical consistency. We introduce a Structured 4D Latent Predictive Model, which predicts the evolution of a scene's 3D structure in a structured latent space conditioned on observations and textual instructions. Our representation encodes the scene holistically and can be decoded into diverse 3D formats, enabling a more complete and 3D consistent scene understanding. This structured 4D latent predictive model serves as a planner, generating future scenes that are translated into executable actions by a goal‑conditioned inverse dynamics module. Experiments demonstrate that our model generates futures with strong visual quality, substantially better 3D consistency and multi‑view coherence compared to state‑of‑the‑art video‑based planners. Consequently, our full planning pipeline achieves superior performance on complex manipulation tasks, exhibits robust generalization to novel visual conditions, and proves effective on real‑world robotic platforms. Our website is available at https://structured‑4d‑model.github.io/.
Authors:Tatiana Gaintseva, Akshit Achara, Gregory Slabaugh, Jiankang Deng, Ismail Elezi
Abstract:
Text‑to‑image diffusion models power everyday creative tasks, but they still reproduce the demographic biases in their training data. On common prompts such as ``a photo of a nurse,'' ``a photo of a CEO'', they skew their outputs toward one gender, driven by the statistics of training data rather than anything in the text. Existing debiasing methods show promise in narrow settings but require retraining, batch‑level control, or prompt‑specific tuning, limiting their scalability. We propose \emphEquiSteer, a training‑free method that works per sample by steering cross‑attention (CA) activations at inference time. For each target attribute, EquiSteer precomputes steering vectors from contrastive prompts. Then at generation time, a prompt‑aware gate leaves attribute‑specific prompts untouched, while for neutral ones it clears existing attribute signals from the CA activations and injects a target attribute. Across SD‑1.5, SD‑2.1, SDXL, and SANA, EquiSteer reduces the average parity gap by up to 87%, with minimal effect on image quality and text‑image alignment. Code is available at \hrefhttps://github.com/Atmyre/EquiSteerhttps://github.com/Atmyre/EquiSteer.%
Authors:Eunsung Cha, Hyunjoon Lee, Jaesik Park
Abstract:
Open‑vocabulary 3D Gaussian segmentation is challenging because it requires language understanding for diverse queries and accurate separation of Gaussians along object boundaries. Prior approaches either embed language knowledge into individual Gaussians to improve query responsiveness or optimize per‑Gaussian instance features to encode object identity. However, these strategies may produce noisy Gaussian segmentations or rely on cost‑inefficient per‑scene optimization. We propose PairGS, a framework that reframes Gaussian segmentation as modeling pairwise relations between Gaussians. 3D Gaussian representations provide rich signals for relation estimation, such as view contribution weights and multi‑view mask evidence. By leveraging these cues, PairGS explicitly constructs a relation graph for segmentation without a heavy optimization process. PairGS first proposes sparse edge candidates using low‑dimensional descriptors, computes precise pairwise affinities only on those candidates, and builds a hierarchical cluster tree for multi‑granular querying. It achieves state‑of‑the‑art results on open‑vocabulary 3D Gaussian segmentation benchmarks, while the fast variant is 50x faster than optimization‑based instance‑feature approaches.
Authors:Sihyeon Lee, Hojeong Lee, Sungwon Woo, Chengpo Yan, Suman Banerjee, Seyeon Kim
Abstract:
We present NPUsper, a live transcription system that makes Whisper efficient on mobile NPUs by eliminating redundant computation. To avoid the heavy padding used by prior streaming systems, NPUsper detects hallucinated tokens online from temporal patterns in decoder cross‑attention, allowing each inference round to process short audio inputs with minimal carryover. For efficient mobile‑NPU execution, we propose controlled unrolling, which executes autoregressive decoding as K‑step chunk graphs, removing unnecessary KV‑cache computation and reducing graph‑dispatch overhead. NPUsper achieves up to 4.84x lower per‑word latency, up to 33.2x lower time‑to‑first‑token (TTFT), and up to 88.64% lower average power consumption compared with baselines, while maintaining comparable transcription accuracy. The code is available at https://github.com/npusper/NPUsper.
Authors:Zhishang Xiang, Zerui Chen, Yunbo Tang, Zhimin Wei, Ruqin Ning, Yujie Lin, Qinggang Zhang, Jinsong Su
Abstract:
Memory has emerged as a cornerstone of modern LLM‑based agents, supporting their evolution from single‑turn assistants to long‑term collaborators. However, memory is not always beneficial: retrieved memories often induce a critical issue of sycophancy, causing agents to over‑align with the user at the cost of factual accuracy or objective reasoning. Despite this emerging risk, existing memory benchmarks primarily evaluate whether memories are correctly stored, retrieved, or updated, while overlooking how retrieved memories influence downstream reasoning and decision‑making. To bridge this gap, we propose MemSyco‑Bench, a comprehensive benchmark for evaluating memory‑induced sycophancy in agent systems. MemSyco‑Bench measures when memory should influence a decision and how valid memory should be used. Specifically, it covers five tasks that assess whether agents can reject memory as factual evidence, respect its applicable scope, resolve conflicts between memory and objective evidence, track memory updates, and use valid memory for personalization. All related resources are collected for the community at https://github.com/XMUDeepLIT/MemSyco‑Bench.
Authors:Dianyu Wang, Yidan Zhang, Peirong Zhang, Xuyang Li, Xiaoxuan Liu, Lei Wang
Abstract:
Recent multimodal large language models (MLLMs) have shown strong cross‑modal understanding and coordinate generation abilities in visual grounding. However, transferring these abilities to remote sensing visual grounding (RSVG) remains challenging. High‑resolution remote sensing images usually cover large‑scale scenes, where targets are often extremely small and surrounded by numerous visually similar distractors. Meanwhile, queries often contain multiple clues, such as reference objects, spatial relations, and target attributes. Existing MLLM‑based methods usually formulate RSVG as one‑step coordinate generation, which may lead to unstable predictions for small‑object localization and complex queries. To address these challenges, we propose GeoSearcher, which reformulates RSVG as an anchor‑guided progressive reasoning process and realizes it through two coupled stages: Anchor‑Centric Reasoning Supervised Fine‑Tuning (ACR‑SFT) and Process‑Faithful Group Relative Policy Optimization (PF‑GRPO). In ACR‑SFT, anchor‑centric reasoning data are used to teach the model to represent key visual clues as anchors and progressively integrate location, relational, and attribute clues around them. In PF‑GRPO, Process‑Aware Reward (PAR) and Reasoning‑Informative Sample Selector (RISS) further optimize this reasoning behavior by jointly evaluating key reasoning steps and target localization, while focusing training on samples that are more beneficial for improving progressive reasoning. Through this design, GeoSearcher transforms large‑scale visual search into a more constrained local reasoning process. Extensive experiments on DIOR‑RSVG, OPT‑RSVG, and VRS‑Bench show that GeoSearcher outperforms existing state‑of‑the‑art methods. The project will be released at https://github.com/wangdianyu954‑xixi/GeoSearcher.
Authors:Rocio Jimenez-Villen, Ziwei Xu, Ying Chen, Oscar Araque, Ryutaro Ichise
Abstract:
Financial markets evolve in response to real‑world events reported in news, yet these drivers often remain implicit in text. To better explain market dynamics, event‑market relations must be explicitly modeled through factual, company‑centric, and environment‑aware knowledge graphs. We present FinKG‑News, a framework that automatically constructs such graphs by extracting news events as anchors linked to companies. Using FinKG‑News as grounded evidence that integrates events, news, and company data, we develop an in‑context learning architecture for credit risk report generation across three core financial dimensions. Automatic and human evaluations show that automated hallucination detection and quality assessment remain unreliable, making expert judgment indispensable. Our approach consistently outperforms baselines, improving quality by 19%‑34% while reducing hallucinations. The source code and project resources are publicly available at: https://github.com/ichise‑laboratory/FINKG‑news.
Authors:Yahya Aalaila, Gerrit Großmann, Sebastian Vollmer
Abstract:
Spatiotemporal point processes (STPPs) model event data in continuous time and space, with applications in mobility, epidemiology, and public safety. Recent neural STPPs span expressive intensity models, conditional density models, continuous‑time latent dynamics, normalizing‑flow spatial decoders, and score‑based generative mechanisms. Yet comparison remains fragile because implementations differ in preprocessing, coordinate normalization, splits, likelihood conventions, and evaluation protocols. We present SEAHORSE, a unified framework for reproducible STPP experimentation. SEAHORSE formalizes neural STPPs through a common encode‑evolve‑decode interface and trains, tunes, and evaluates every model family under a single executable benchmark protocol with raw‑coordinate likelihood reporting. This enables fair comparisons but, more importantly, controlled diagnostic studies. We pair SEAHORSE with HawkesNest, a synthetic stress‑test suite, and show that increasing event‑pattern complexity exposes each family's inductive bias, degrading some models sharply and leaving others stable. Code: https://github.com/YahyaAalaila/seahorse.
Authors:Nils Neukirch, Martin Maurer, Nils Strodthoff
Abstract:
Radiomics is the established approach for CT‑based lung cancer phenotyping, yet comparisons with foundation models rarely isolate contributions of feature extractor, classification head, and segmentation choice, or test cross‑cohort robustness. We benchmark five feature extractors (Curia, Curia‑2, DINOv3, Radiomics2D, Radiomics3D), seven classification heads (TabPFN, TabICL, XGBoost, CatBoost, Random Forest, logistic regression, Ridge), and three segmentation regimes on five tasks: tumor volume and stage classification, 2‑year survival prediction, histology classification, and age prediction. Models are trained on LUNG1 (n=338) and evaluated on an internal test set (n=84) and the external LUNG2 cohort (n=211), with worst‑case cross‑cohort performance as the primary metric. The dominant design factor is task‑dependent: segmentation drives volume and stage classification, while classifier choice drives survival, histology, and age prediction. Radiomics is competitive for tumor volume, tumor stage and survival (partly due to label‑derivation effects for the former); Curia variants reach comparable peak scores for survival; DINOv3 falls slightly short across tasks. Patch and slice aggregation have negligible impact. We recommend Curia with tumor segmentation and a CatBoost head as a safe default, achieving the best mean rank across the three primary clinical tasks, though task‑specific selection consistently outperforms any cross‑task default. When tumor delineations are unavailable, Curia‑2 with lung segmentation and logistic regression offers a competitive alternative. All pipelines use a two‑stage design suited to small cohort sizes where end‑to‑end fine‑tuning would risk overfitting.
Authors:Geunhyuk Youk, Jeonghyeok Do, Dayeon Kim, Jihyong Oh, Munchurl Kim
Abstract:
Diffusion models have significantly advanced video super‑resolution (VSR) but remain largely constrained to fixed upsampling scales. Conversely, while coordinate‑based arbitrary‑scale VSR methods offer scale flexibility, they inherently suffer from severe over‑smoothing at large scaling factors. Integrating generative priors with continuous decoding is promising but currently hindered by severe temporal flickering caused by the stochasticity of diffusion sampling. To address this, we propose AVSR‑Diff (Arbitrary‑scale Video Super‑Resolution with Diffusion), a novel decoupled framework that separates scale‑agnostic latent denoising from continuous coordinate rendering, effectively avoiding computationally heavy resolution‑specific sampling. Our approach introduces a Temporally‑Gated Feature Recurrence (TGFR) module to extract strictly aligned, temporally consistent latent priors. Furthermore, we design a continuous video VAE decoder incorporating a Scale‑Aware Fourier Refinement (SAFR) module to dynamically adapt frequency components to any target scale. Extensive experiments demonstrate that AVSR‑Diff consistently preserves high‑frequency details and strong temporal stability across various scales, surpassing state‑of‑the‑art arbitrary‑scale baselines. Remarkably, our framework outperforms recent fixed‑scale generative models even on their native resolution.
Authors:Jun Peng, Baiyang Song, Jie Li, Hui Li, Yiyi Zhou, Rongrong Ji, Yonghong Tian
Abstract:
Video understanding is often plagued by severe temporal redundancy, where processing dense frame sequences is both semantically inefficient and computationally expensive. This challenge is further amplified when only a small subset of frames is truly relevant to the given query. In this paper, we propose a Query‑ and Content‑Aware (QCA) keyframe selection framework that can select a compact yet information‑rich set of frames from long videos. QCA first partitions the video into temporal segments and estimates the information contribution of each segment by jointly modeling query relevance and content deviation, and dynamically allocates keyframe budget to each segment. Within each segment, QCA anchors on the most query‑relevant frame and iteratively incorporates additional frames to maximize diversity while maintaining high semantic relevance to the query. Crucially, our method requires no additional training and can be seamlessly integrated into existing Video‑LLMs. Extensive experiments across multiple long video understanding benchmarks demonstrate that our proposed approach achieves state‑of‑the‑art performance and has strong generalization ability. For instance, QCA achieves 67.8% on LongVideoBench using 128 frames, while GPT‑4o achieves 66.7% using 256 frames. Our codes are available in \hrefhttps://github.com/hktk07/QCAGitHub.
Authors:Yiwen Xing, Philip Beaucamp, Joyraj Chakraborty, Afrah Farea, Yuanzhe Jin, Saiful Khan, Gennady Andrienko, Natalia Andrienko, Min Chen
Abstract:
Visual analytics (VA) plays an increasingly important role in supporting machine learning (ML) workflows. In the field of visualization, such approaches and techniques are referred to as VIS4ML. While ML models are mostly learned automatically, the corresponding ML workflows receive a variety of human inputs, such as data labelling, feature engineering, model architecture designing, hyper‑parameter tuning, and so on. In this work, we surveyed over 200 VIS4ML papers to gain an understanding of how humans inject their knowledge into ML workflows through interactive visualization. We collected a corpus of VIS4ML papers from the IEEE VIS conferences in the past decade. We developed a coding scheme to facilitate the literature research from four perspectives: characteristics of ML, visualization, interaction, and actions. The analysis of the coded dataset allows us to observe different pathways that transfer human knowledge to ML workflows via interactive visualization. Building on the analysis, we explain the phenomena of VIS4ML using the conceptual model that views VA as model building and the information‑theoretic cost‑benefit analysis that reasons VA as for optimizing ML workflows. This work provides unequivocal evidence showing the merits of using VA in ML workflows. The full list of surveyed papers, along with all analysis results and figures, is available at https://vis4ml4hd.github.io/ml‑knowledge‑inject‑va/.
Authors:Haijie Yang, Zhenyu Zhang, Yixuan Dong, Jianjun Qian, Jian Yang
Abstract:
Audio‑driven talking head synthesis has achieved impressive progress in lip synchronization and visual quality, yet generating expressive emotional avatars with controllable intensity remains challenging, especially under real‑time constraints. In this paper, we present GaussianEmoTalker, an audio‑driven framework for real‑time emotional talking head synthesis based on 3D Gaussian Splatting. Instead of directly predicting the final emotional avatar from speech, we formulate emotional animation as a neutral‑to‑emotional residual deformation problem. GaussianEmoTalker first constructs an identity‑specific neutral talking space with GaussianBlendshapes, which provides high‑fidelity Gaussian attributes and phoneme‑synchronized neutral motion. It then predicts an emotion‑conditioned residual deformation by combining mesh displacement cues, audio features, emotion categories, and intensity encodings. To fuse these heterogeneous signals, we introduce a spatial‑audio‑emotion attention module that estimates the offsets of Gaussian attributes for expressive and temporally stable rendering. Extensive experiments demonstrate that GaussianEmoTalker achieves competitive video quality, accurate lip synchronization, controllable emotional expression, and real‑time rendering compared with recent emotional talking head methods. Our project page is available at https://njust‑yang.github.io/GaussianEmoTalker.github.io/
Authors:Alexander Chemeris, Ming Jin, Randall Balestriero
Abstract:
Time series are central to modern data mining applications, from industrial telemetry and server metrics to finance and physiology, yet time‑series self‑supervised learning often depends on view and augmentation choices that encode domain‑specific invariances. We study how an SSL recipe behaves when its method‑specific configuration is reused unchanged after the pretraining signal family changes, framing this as a fixed‑recipe stress test rather than a comparison against optimally tuned methods. We introduce Latent Euclidean Next‑Embedding Prediction Architecture (LeNEPA), a no‑augmentation next‑latent‑token objective with a causal backbone. LeNEPA replaces the stop‑gradient/EMA stabilization used by vanilla NEPA with SIGReg‑based isotropy regularization and computes the predictive loss in a lightweight projected space that is discarded for evaluation. We compare LeNEPA with an ECG‑tuned JEPA recipe under a fixed‑horizon frozen‑probe protocol on PTB‑XL and Diag, a synthetic diagnostic corpus generated with Aionoscope. Both methods are retrained independently on each dataset while keeping their method‑specific recipes unchanged. In this protocol, the ECG‑tuned JEPA recipe is strong in‑domain on PTB‑XL but weaker when reused unchanged on Diag, whereas LeNEPA preserves useful frozen‑probe gains on both datasets. Learning curves suggest faster early representation acquisition: LeNEPA reaches 80% of its final AUROC/AUPRC gain after 2‑‑5k updates, compared with 5‑‑10k updates for the faster JEPA readout. As a separate external frozen‑encoder check, a CauKer‑pretrained LeNEPA variant reaches 77.65% mean UCR‑128 Random‑Forest accuracy in a single‑seed, best‑checkpoint run, within 1.16 points of Mantis and within 0.24 points of MOMENT (77.89%). Overall, the results support no‑augmentation latent prediction as a useful candidate recipe for low‑retuning time‑series SSL.
Authors:Alexander Chemeris, Ming Jin, Randall Balestriero
Abstract:
Time‑series models are often evaluated by what they can forecast or classify, but those scores do not show whether their representations preserve the process state a user may want to inspect: event timing, phase, amplitude, frequency, or regime variables. We introduce Aionoscope, a generator‑based diagnostic tool for debugging latent‑state accessibility in frozen time‑series representations. Aionoscope separates process generation from observation rendering, producing seeded synthetic streams with exact categorical and dense labels across mixture complexity and nuisance variation. We instantiate Aionoscope as Primitive Process Mixtures and evaluate 37 model‑plus‑adapter systems with a common pooled linear‑probe protocol. The main result is a mismatch between coarse and fine‑grained accessibility. Most systems make component presence easy to recover, but expose dense process state much less reliably: the highest observed dense‑probe row reaches 0.689 mean masked R^2, while a dense‑feature oracle reaches 0.999. This is the failure mode Aionoscope is designed to surface: a representation can look informative at the level of "what kind of signal is present" while hiding the timing, phase, amplitude, frequency, or regime variables needed for debugging.
Authors:Christopher Lindenberg, Kashyap Chitta
Abstract:
World models can enable Model Predictive Control (MPC), but this requires dynamics prediction that is both fast enough for online use and expressive enough to represent uncertain futures. Diffusion models offer a natural mechanism for modeling uncertain dynamics, yet their iterative inference procedure makes them difficult to use for low‑latency latent planning. We bridge this gap with Value Diffusion World Models (Valdi), combining end‑to‑end online training for MPC with a latent diffusion dynamics model. In preliminary experiments on the CarRacing environment, we show that Valdi, using a single diffusion step at both training and inference, matches a deterministic MLP baseline. Our experiments expose a trade‑off between predictive multimodality and control performance in this setup. Code is available at https://github.com/Kit115/ValueDiffusionWorldModels.
Authors:Xinyi Shang, Peng Sun, Bei Shi, Zixuan Wang, Tao Lin
Abstract:
Recent advancements in scaling dataset distillation rely heavily on decoupled information extraction pipelines, comprising SQUEEZE, RECOVER, and RELABEL stages. Despite their scalability to large‑scale datasets, these methods suffer from prohibitive computational overhead and poor cross‑architecture generalization. In this paper, we reveal the root cause of these bottlenecks: the implicit dual‑compression process, from data to model and back to images, inherently induces severe information loss. Crucially, we empirically and theoretically demonstrate that this loss creates a distribution shift that fundamentally compromises the widely adopted RELABEL strategy, transforming the pre‑trained model into an unreliable labeler that yields sub‑optimal labels. To overcome these critical flaws, we propose CIM, a novel, metric‑driven framework that abandons the flawed dual‑compression paradigm. Instead, CIM explicitly quantifies and minimizes the information gap between the original and synthetic datasets. By directly aligning the data distributions, our approach ensures high‑fidelity information condensation and inherently satisfies the prerequisites for effective relabeling. Extensive experiments demonstrate that CIM establishes a new state‑of‑the‑art. Notably, it distills ImageNet‑1K at an IPC=10 in merely 80 minutes on a single RTX‑4090 GPU, achieving an unprecedented 48.7% Top‑1 accuracy on ResNet‑18 and significantly outperforming previous SOTA approaches, such as NRR‑DD and DELT, by 2.6% and 2.9%, respectively. Our code is available at https://github.com/LINs‑lab/CIM.
Authors:Youwei Pang, Xiaoqi Zhao
Abstract:
Evaluation metrics are central to binary target segmentation because they determine how progress is measured, compared, and interpreted. In this paper, target denotes the task‑defined positive region to be segmented rather than a generic foreground object. It may be salient, camouflaged, transparent, glass‑like, mirror‑like, shadow‑like, lesion‑like, or defined by other application‑specific semantics. We treat existing metrics as compositions of modular design choices rather than isolated formulas. The proposed framework decomposes each metric into five stages covering prediction representation, target extraction, target matching, score computation, and metric reporting. We use this framework to analyze representative metrics and show how newer metrics address specific limits in earlier protocols. The stage choices keep each metric's assumptions visible. We then discuss the design space opened by the framework and its implications for task‑aware evaluation protocols. Reference code is available at https://github.com/lartpang/PySODMetrics.
Authors:Kangmin Seo, Sangeek Hyun, MinKyu Lee, Jae-Pil Heo
Abstract:
Recent advances in neural rendering have established 3D Gaussian Splatting (3DGS) as a highly efficient representation for novel view synthesis, enabling fast training and real‑time rendering with strong fidelity. However, when supervision is limited to sparse input views, 3DGS tends to overfit to the observed images and generalize poorly to unseen viewpoints. We address this challenge from the perspective of flat minima (FM) optimization, which seeks solutions that remain stable under small parameter perturbations. Viewing Gaussian parameters as trainable weights, we adapt FM principles to the geometric and dynamic nature of 3DGS with a lightweight training framework. Our method regularizes optimization with controlled Gaussian perturbations that account for each Gaussian's anisotropy and the training progress, preserving fine details while improving robustness to sparse‑view overfitting. To further stabilize this flat minima optimization process, we introduce periodic reinitialization, which temporarily returns non‑positional parameters to their initial states for a short window. Together, these techniques integrate seamlessly into existing 3DGS pipelines without architectural changes. Experiments on LLFF and Mip‑NeRF360 datasets demonstrate improved quantitative metrics and perceptual quality under sparse‑view supervision, producing reconstructions that are sharper, more stable, and better generalized to novel viewpoints.
Authors:Omer Sela, Inbar Huberman-Spiegelglas, Michael Rotman, Sagie Benaim, Avi Ben-Cohen
Abstract:
Controlling the motion of multiple objects in image‑to‑video (I2V) generation requires preserving object identities while enforcing adherence to distinct target trajectories. This becomes particularly challenging as the number of objects increases and their paths intersect or occlude one another. Existing approaches entangle multiple trajectories within a shared, dense conditioning signal, making object‑level correspondence difficult to preserve in crowded scenes. We depart from this paradigm and enforce a strict, per object spatial constraint that isolates instances independently. Our method, TrajLoc, achieves this directly within the attention layers by substituting the cross‑attention weights of each object token with a Gaussian heatmap centered on its target location at every frame. The same per object token interface carries trajectory and depth through a learned embedding and preserves identity by encoding first frame appearance in place of an object token. Evaluations across six datasets, featuring up to 20 simultaneously controlled objects and out of distribution real world scenes, demonstrate that our method consistently improves both visual fidelity and trajectory adherence. Applied to two architecturally distinct backbones (CogVideoX 5B and WaN 2.1 14B), our approach achieves average gains of +4.3 dB PSNR and a 51% reduction in trajectory end point error compared to the strongest baselines. Project page: https://sela‑omer.github.io/traj‑loc/
Authors:Xiaoxiong Zhang, Xiong Zeng, Wei Zhang
Abstract:
World models are increasingly used in embodied intelligence and generative simulation, yet their scope remains ambiguous across communities. This tutorial presents a design‑space view of world models as action‑conditioned predictive models that estimate the future evolution of task‑relevant observations or states. We categorize existing methods into observation‑space and state‑space world models, comparing their trade‑offs in visual fidelity, spatial structure, physical interpretability, and control usability. We further introduce world action models, which connect predicted futures with executable robot actions, and summarize four representative paradigms: imagine‑then‑execute, video‑feature‑conditioned action prediction, joint video‑action modeling, and auxiliary video prediction for policy learning. The goal of this tutorial is to clarify the conceptual scope of world (action) models and provide a structured taxonomy for embodied prediction and control.
Authors:Andrea Sanchietti, Riccardo Marin, Bharat Lal Bhatnagar, Yuanlu Xu, Gerard Pons-Moll
Abstract:
While garments are essential for realistic digital humans, their topological variety makes them much harder to model than parametric bodies. Traditional tailoring relies on 2D sewing patterns, yet bridging these patterns to 3D geometry currently requires physical simulations. We present Stitched Embeddings, the first simulation‑free framework to unify 3D garment reconstruction and sewing pattern inference within a single bidirectional latent space. By leveraging the geometric priors of a pretrained 3D foundation model, our approach overcomes the data scarcity typically associated with high‑quality garment modeling. We propose to use the BoxMesh as a critical intermediate representation to align 2D panels into 3D configurations without the computational overhead of a simulator. This architecture achieves state‑of‑the‑art accuracy in pattern reconstruction while significantly improving efficiency. Furthermore, our differentiable pipeline enables novel applications, including pattern recovery from meshes and 3D editing from 2D patterns. Finally, this work provides a scalable link between neural 3D vision and the physical garment manufacturing pipeline. Project Page: https://andreus00.github.io/stitchedembeddings
Authors:Yaofei Duan, Yuhao Huang, Tianyu Zhang, Yuan Gao, Luyi Han, Xin Wang, Xinyu Xie, Xinglong Liang, Chunyao Lu, Muzhen He, Patrick Pang, Yue Sun, Ning Mao, Tao Tan, Ritse Mann
Abstract:
Neoadjuvant chemotherapy (NAC) response prediction is clinically important for treatment stratification in breast cancer. However, robust pre‑treatment pathological complete response (pCR) prediction remains challenging due to insufficient cross‑modal modeling, multicenter imaging heterogeneity, and weak evidence‑grounded interpretability. We propose ClinRAG‑GRAPH, a Clinically informed Retrieval‑Augmented Generation Graph framework, for pre‑treatment pCR prediction from DCE‑MRI, structured clinical variables, and biopsy‑derived pathological biomarkers. ClinRAG‑GRAPH constructs an intra‑patient clinical‑prior graph and applies a prior‑guided relation‑aware graph convolutional network for structured multimodal representation learning. To improve cross‑center robustness, we introduce a dual‑branch domain‑adversarial learning strategy to suppress protocol‑related MRI bias while preserving pCR‑relevant features. To enhance interpretability, we further incorporate large language model (LLM)‑driven subgraph RAG module that retrieves clinically analogous historical cases and integrates retrieved evidence for pCR inference. We assemble a large‑scale multicenter NAC breast cancer cohort for extensive validation, drawing from two public sources and three in‑house centers.Results show that ClinRAG‑GRAPH achieves AUCs of 0.815 on the internal test set and 0.774/0.712 on two external test sets, demonstrating robust pre‑treatment pCR prediction across centers. The code is available at the anonymized https://github.com/miccai26‑1181/ClinRAG‑GRAPH.
Authors:Çağrı Eser
Abstract:
Recognizing jazz standards from audio is a challenging form of tune‑level music retrieval: different performances of the same standard may vary in tempo, key, arrangement, instrumentation, improvisational content, and even whether the head melody is present. We study this problem using a curated subset of the Jazz Trio Database designed for cross‑performance standard recognition. We compare a from‑scratch trained Harmonic CNN baseline against frozen pretrained music representations from recent music understanding foundation models, using both supervised probing and nearest‑neighbor retrieval. Our results suggest that from‑scratch spectrogram models overfit strongly to training performances, while pretrained embeddings provide better top‑k results but are sensitive to performer identity, which can be partially reduced with a lightweight contrastive projection. Our findings motivate jazz standard recognition as a useful stress test for music representation models and as a step toward retrieval‑based standard identification. Project page: https://github.com/cagries/tipofmyear.
Authors:Changsheng Lu, Yuxin Chen, Haokun Gui, Rong Wang, Jie Yang, Harry Yang, Anton van den Hengel, Jiaya Jia
Abstract:
With the emergence of various pre‑trained vision and language models, computer vision is shifting from narrow‑domain to open‑domain recognition. The construction of a more powerful yet general keypoint detection (GKD) model to support diverse tasks has become increasingly important in the field. To this end, we firstly present a large‑scale unified keypoint dataset called MegaKPT. The dataset is composed of over 1.3 million diverse object instances from twenty‑nine existing datasets, and enjoys high‑quality unified annotations with keypoint text descriptions. Based on MegaKPT, we develop GKDT, a simple, flexible and powerful DINOv3 based Transformer model for General Keypoint Detection. Our GKDT supports visual prompts, text prompts, or both. To enhance model training, we also propose a suite of useful strategies such as mix‑modal prompted training and dynamic importance sampling. By testing over 22 test sets with seen or unseen objects, our single GKDT model shows strong performance and generality in detecting keypoints on broad categories, with most categories over 90% PCK@0.1 accuracy, offering high practical applicability to real‑world problems. The dataset, models, and codes will be released at https://github.com/AlanLuSun/General‑Keypoint‑Detection.
Authors:Minmin Wu
Abstract:
Enterprise AI agents are useful for internal analysis, audit, compliance review, and operational investigation, but they create a difficult authorization problem. A manager or data owner may approve a business task, while the agent later generates open‑ended SQL below the application layer. Existing systems help identify agents, delegate authority, govern data products, or enforce database policy, but they do not directly turn an approved enterprise task into a bounded database execution context. SessionBound fills this gap. It turns approved enterprise tasks into short‑lived, budgeted, and auditable database sessions for AI agents. A control plane defines task templates, accepts task applications, records approvals, assigns budgets, and issues signed task tokens. A database runtime, SessionBoundDB, binds a token to a session and enforces safe views, row scope, denied fields, operation limits, query budgets, disclosure budgets, and receipts. The database does not rely on an LLM to decide whether a query is safe. The agent may generate SQL freely, but each attempt must stay inside the approved boundary. A PostgreSQL prototype passed a 24‑scenario validation suite. Microbenchmarks show p50 SessionBound execution around 1.4‑‑1.5 ms versus raw PostgreSQL p50 around 0.052‑‑0.074 ms on small synthetic queries: high relative overhead, but low absolute latency.
Authors:Rusi Chen, Yuhao Huang, Hongyuan Zhang, Chao Tian, Shunan Ji, Yuhan Zhang, Dong Ni
Abstract:
Accurate detection of end‑systole (ES) and end‑diastole (ED) frames is fundamental to echocardiographic assessment. Existing methods are typically developed in a view‑specific manner, depend on auxiliary annotations or intensive visual modeling, which limits their generalizability. In multi‑view modeling, keyframe detection is driven by shared cardiac motion, yet large appearance differences and motion patterns make unified modeling challenging. To address these issues, we propose FrameONE, a unified end‑to‑end framework for multi‑view echocardiographic keyframe detection. FrameONE introduces a Hierarchical Motion Modeling strategy: an intra‑view multi‑task learning reduces appearance bias and promotes motion‑focused representations within each view; an inter‑view general motion learning module further separates view‑agnostic dynamics from view‑specific patterns, enabling shared yet flexible motion representation learning across views. Extensive experiments on 25,872 videos spanning four standard views demonstrate that FrameONE achieves state‑of‑the‑art keyframe detection accuracy with strong cross‑view generalization. Code is available at https://github.com/szuboy/FrameONE.
Authors:Le Ou, Xiliang Zhu, Huanwen Liang, Wenxiong Pan, Yuhao Huang, Yuxiang Deng, Xuan Sheng, Hong Yin, Juhua Xiao, Xin Zhou, Dong Ni
Abstract:
Accurate fetal birth weight (FBW) estimation shortly before delivery is clinically valuable yet challenging due to its reliance on operator expertise, particularly in low‑resource settings. To reduce this reliance, we study near‑term birth‑weight regression from blind‑sweep ultrasound (US) videos acquired within 48 hours prior to delivery, with post‑delivery weighing as ground truth. Accordingly, we propose a foundation model‑driven key anatomy frame selection framework that enables accurate FBW regression despite the absence of plane constraints in blind sweeps. Our highlights are as follows: (1) We believe this is the first work to estimate FBW using blind‑sweep US videos, enabling operator‑independent assessment. (2) An Anatomy‑Guided Frame Selection module equipped with a vision‑language foundation model is proposed for keyframe collection in unconstrained sweeps. (3) A Redundancy‑Aware Feature Compression module is designed to compress frame features while preserving task‑relevant information, alleviating temporal redundancy. Extensively validated on prospectively collected data from 839 patients, our method achieves an MAE of 161.3 g, with 90.23% and 100% of cases falling within 10% and 15% absolute percentage error, outperforming typical Hadlock estimation and strong competitors. Codes are available at https://github.com/ouleoule/BlindSweep‑EBW.
Authors:Mark Russinovich, Ram Shankar Siva Kumar, Ahmed Salem
Abstract:
Large language models can generate polished scientific text that includes unsupported claims, allowing hallucinations to enter the archival record. Assessing this risk via technical statements is difficult and often requires expert judgment, but citations provide a more auditable surface: a reference either resolves to a real scholarly work with compatible authorship, or it does not. We measure citation hallucination in peer‑reviewed proceedings using a conservative definition limited to identity‑level failures: non‑existent works and substantial author‑list mismatches. We explicitly exclude ordinary bibliographic drift (e.g., venue/year differences, publication‑status updates, minor name variants). To audit citations at scale, we build RefChecker, a verification pipeline that resolves bibliography entries against multiple bibliographic sources and escalates unresolved cases to web‑search re‑verification. We apply RefChecker to accepted camera‑ready papers from ICLR, ICML, NeurIPS, and USENIX Security. Hallucinated citations have entered the archival record. While reference‑level rates are usually below 1%, proceedings are large enough that paper‑level failures are visible: in 2025, roughly one in twenty NeurIPS and USENIX Security papers contains at least two likely hallucinated academic‑paper‑like references under our strict definition. We also observe post‑ChatGPT increases in several venues, including a tail of papers with 5+ failures in a single bibliography, and likely hallucinated citations even among award‑winning papers. These results suggest peer review alone does not reliably enforce citation integrity, yet auditing is tractable (about 0.04 per paper in one venue‑scale scan). We open‑source RefChecker for routine, reproducible citation verification before publication (https://github.com/markrussinovich/refchecker).
Authors:Zhaowen Zhu, Li Zhang, Yujie Chen, Tian Zhang, Yingjie Wang, Mingxia Zhan
Abstract:
Self‑Supervised Monocular Depth Estimation (MDE) has garnered attention in recent years due to its independence from ground truth. However, most existing models are limited to a single scale and exhibit considerable performance degradation in complex driving environments. Networks specifically designed to handle dynamic traffic participants tend to be overly complex, hindering their deployment on resource‑constrained automotive edge devices. To address these limitations and move towards robust driving perception, we propose FlexDepth, a scale‑driven and flexible family of self‑supervised MDE models tailored for challenging road scenarios. FlexDepth employs a two‑stage static‑dynamic decoupled training strategy, enabling the independent assessment of confidence for both static backgrounds and dynamic road objects. Furthermore, it introduces a meticulously designed Scale‑Driven Decoder (SDD) to dynamically select components based on scale size, facilitating efficient feature fusion and the output of high‑precision depth maps. Extensive experiments on standard driving benchmarks demonstrate that without any auxiliary information, our model achieves state‑of‑the‑art performance across arbitrary scales with minimal computational overhead. Our smallest model, Flex‑Nano, requires only 0.7 GFLOPs and achieves 37.6 FPS on mobile platforms, ensuring reliable real‑time perception while maintaining excellent zero‑shot generalization. Our source code is avalible: https://github.com/startnew/flexdepth
Authors:Madhulatha Mandarapu, Sandeep Kunkunuru
Abstract:
Graph approximate‑nearest‑neighbor (ANN) indexes (HNSW, DiskANN/Vamana) lose recall under insert/delete churn, because deletions orphan the greedy‑search paths that route through removed nodes. Production systems restore navigability by repairing the graph on a fixed schedule (consolidate every X operations). We ask whether triggering local edge repair on a measured navigability‑degradation signal, rather than a blind clock, spends a fixed repair budget better. On two real ANN datasets (SIFT‑128 and Fashion‑MNIST‑784) under a controlled bursty churn stream, and comparing repair policies at matched amortized repair budget (equal consolidation count), signal‑triggered repair Pareto‑dominates fixed‑cadence repair. The gain is concentrated on worst‑case (tail) recall at scarce budget: at roughly one consolidation it improves the minimum recall@10 by +0.014 (SIFT) to +0.050 (Fashion‑MNIST) across four stream seeds, with 95% confidence intervals excluding zero, while the mean‑recall gain is small (<0.005). The advantage follows a clean drift‑severity gradient ‑‑ larger for sparser, more fragile graphs ‑‑ and fades to parity when the index is robust or budget is ample. A cheap probe‑recall signal is a valid, leading indicator of true recall (Spearman rho ~= 0.95). We contribute the mechanism, a budget‑matched evaluation protocol that separates repair scheduling from repair spend, and an open, reproducible churn‑repair harness. We deliberately do not claim a mean‑recall improvement or a new index; a recall‑versus‑repair‑cost bound and data‑distribution‑drift coupling are left as future work.
Authors:Tianhong Zhou, Mingyang Han, Boyu Li, Yuxuan Jiang, Jiaxin Ye, Dongxiao Wang, Haoxiang Shi, Kunpeng Wang, Jun Song, Cheng Yu, Bo Zheng
Abstract:
Audio‑visual feature extraction is a fundamental component of multimodal understanding and generation tasks. However, existing evaluation protocols for feature extraction models exhibit dimensional bias, typically focusing on either semantic matching or temporal offset detection. Moreover, their data construction remains coupled, preventing independent assessment of temporal and semantic consistency. We propose AV‑SyncBench, the first benchmark to fully separate temporal and semantic evaluation for audio‑visual synchronization. Built from in‑the‑wild videos, it spans Voice, Music, and Sound across 10 scenarios and 5 challenge tasks. Data are automatically filtered and manually verified to ensure on‑screen sound sources. The benchmark contains 3,269 videos and 38,390 samples, and we evaluate five representative models to quantify feature quality for alignment and downstream tasks. The code and dataset are available at: https://fgt7t6g.github.io/AV‑SyncBench.
Authors:Jaehoon Yoo, Wonjung Kim, Floor Eijkelboom, Chanhyuk Lee, Nicholas M. Boffi, Seunghoon Hong, Jinwoo Kim
Abstract:
Self‑conditioning is a core technique that enhances continuous flow‑based language models, where the model learns to denoise generated text by conditioning on its own denoising estimate. While empirically successful, its performance improvements are poorly understood. Moreover, there is growing interest in the use of few‑step generators based on flow maps, for which how to leverage self‑conditioning is unclear. Here, we show that flow language models with self‑conditioning solve a fixed‑point iteration that bootstraps the performance of the learned denoiser. We use this viewpoint to formulate fixed‑point flows, a two‑dimensional class of self‑conditioned flows, where the first dimension represents the flow process and the second represents the fixed‑point iteration. We show that fixed‑point flows define valid flow maps, and show that they can be distilled from self‑conditioned flow models by compressing both fixed‑point iterations and the flow process, the former with fixed‑point distillation and the latter with flow map distillation. Our resulting flow map language model, FMLM^\star, outperforms state‑of‑the‑art self‑conditioned models and few‑step models in one‑ and few‑step generation on OpenWebText. Code is available at https://github.com/Ugness/self‑conditioned‑fmlm.
Authors:Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin, Haoning Wu, Haoyun Liu, Yuzhi Chen, Lulu Zheng, Botai Yuan, Tianlun Li, Mingxin Wang, Dekang Qi, Bin Hu, Wei Mei, Yuze Xuan, Haolong Yang, Yanqing Zhu, Mu Xu, Zhiheng Ma, Xinyuan Chang
Abstract:
Mobile manipulation is a key capability for general‑purpose robots, yet remains challenging for current embodied learning methods. VLA policies are typically reactive and lack explicit world modeling, while existing World Action Models (WAMs) are still poorly aligned with the structure of mobile manipulation: they operate on coarse video chunks, model entangled navigation‑manipulation actions, and train inverse dynamics under supervision that does not match autoregressive inference. As a result, they often miss fine‑grained contact dynamics, suffer from action‑distribution conflicts, and accumulate errors over long‑horizon rollouts. We propose ABot‑M0.5, a new WAM built on the insight that mobile manipulation requires alignment at three levels: temporal granularity, action space, and train‑test consistency. To align temporal granularity, we introduce intermediate latent actions that capture local visual state transitions and serve as an bridging action space between video latents and embodiment‑specific controls. To align action space, we design a dual‑level Mixture‑of‑Transformers architecture that disentangles both modality representations and heterogeneous action subspaces such as base movement and arm manipulation. To align inference conditions, we propose the dream‑forcing training strategy that progressively trains inverse dynamics on model‑predicted videos, improving train‑test alignment and robustness during autoregressive prediction. Experiments on challenging mobile and fine‑grained manipulation benchmarks demonstrate that ABot‑M0.5 achieves state‑of‑the‑art performance in both long‑horizon task success and finegrained control accuracy. These results highlight the critical importance of granularity‑aligned, action‑disentangled, and inference‑consistent world‑action modeling.
Authors:Zhengbo Zhang, Mark He Huang, Zhigang Tu, Ming-Hsuan Yang
Abstract:
Zero‑shot video temporal grounding (VTG) localizes events in untrimmed videos from natural language queries without task‑specific training. Existing methods rely on frame‑query feature matching, which suffices for simple events but struggles with complex multi‑stage queries that require understanding temporal ordering and causal structure ‑‑ a disparity we call the reasoning gap. We propose DART (Difficulty‑Adaptive Routing for Temporal Grounding), which bridges this gap by coupling difficulty‑aware routing with structured reasoning in large vision‑language models. A query‑conditioned Determinantal Point Process (DPP) serves a dual role: selecting diverse, query‑relevant keyframes as temporal evidence, and providing spectral entropy as a difficulty indicator. Simple queries are routed to a Fast path for direct prediction, while complex queries follow a Slow path with Temporal Markup Prompting, which decomposes localization into global event analysis, per‑frame temporal role annotation, and boundary extraction. On Charades‑STA and ActivityNet Captions, DART achieves state‑of‑the‑art zero‑shot performance across both identically distributed and multiple out‑of‑distribution settings, improving mIoU by up to 3.5 points over the strongest baseline while using over 7 times fewer frames. The project homepage is available at https://dart‑vtg.github.io/.
Authors:Taewook Kang, Taeheon Kim, Donghyun Shin, Jonghyun Choi
Abstract:
Vision‑Language‑Action (VLA) models often fail to perform the same learned tasks under environmental shifts, such as changes in camera pose and shifts to a different but similar robot (e.g., from Panda to UR5e). Adapting these models to the shifted environment (i.e., target domain) often requires training on multiple demonstrations for each task, which are costly to collect. To reduce the burden of data curation and training, we propose an analogy‑based method that adapts VLA models under environmental shifts through weight vector arithmetic with domain‑specific information addition, named Domain ARiThmetic (DART). Unlike prior approaches, DART requires collecting only a single demonstration, enabling efficient adaptation. To accurately isolate domain‑specific information for addition, DART performs subspace alignment between singular components in weight vectors to filter out noisy components. In both simulated and real‑world experiments, DART outperforms existing VLA adaptation methods in one‑shot scenarios across diverse visual and embodiment shifts. Code is available at https://github.com/snumprlab/dart.
Authors:Yunsung Lee, Hyeongmin Lee
Abstract:
Training‑free guidance (TFG) steers a pretrained diffusion model toward a desired attribute at inference. To be effective, this guidance must be applied from the earliest, high‑noise steps of sampling. Because its objective (a classifier or energy) is defined on clean images, ε‑ and v‑prediction models must first estimate the clean image \hatx from the noisy state at each step, and the accuracy of that estimate determines how easily guidance drifts off the data manifold. x‑prediction, a recent alternative, outputs the clean image directly, removing this source of error even at high noise. This is our motivation. We provide a theoretical analysis of how each prediction target shapes this accuracy, and introduce guided‑class FID (Child FID), a metric that exposes the manifold damage standard evaluation misses. Experiments on a new fine‑grained bird benchmark and on style transfer confirm that x‑prediction keeps guided samples on the manifold most reliably, making it the strongest foundation for training‑free guidance. Code is available at https://github.com/ManLuML/on‑manifold‑tfg
Authors:Haeun Jeon, Seunghoon Choi, Hyunglip Bae, Yongjae Lee, Woo Chang Kim
Abstract:
Sparse tangent portfolio optimization aims to learn an interpretable, low‑cardinality portfolio in the tangency direction of the mean‑variance frontier. However, the associated cardinality‑constrained formulation is NP‑hard, and standard predict‑then‑optimize pipelines often misalign forecasting accuracy with downstream portfolio quality. We propose an end‑to‑end decision‑focused learning framework that reformulates Sharpe ratio maximization as a Disciplined Parametrized Programming (DPP)‑compliant convex programming layer and replaces discrete selection with a smooth top‑k operator enforcing an exact cardinality k. This enables gradient flow through prediction, asset selection, and re‑optimization, allowing the predictive model to directly optimize portfolio performance. Across four major equity markets, our method achieves competitive and often superior out‑of‑sample Sharpe ratios compared with historical and prediction‑focused baselines, with particularly strong gains in larger asset universes. Our \hrefhttps://github.com/feuerwerksh/Diffble‑card‑SRcode is publicly available.
Authors:Cong Liu, Xiaofang Li, Simon X. Yang
Abstract:
Vision Transformers (ViTs) commonly rely on injected positional mechanisms to address self‑attention's permutation invariance. Motivated by the spatial regularities of natural images, we ask whether spatial organization can be induced from data rather than explicitly injected. Under controlled, matched from‑scratch training, we propose Active Spatial Guidance (Guidance), a training‑only objective that disables positional injection and applies an auxiliary 2D coordinate‑regression loss to the final‑layer patch tokens. The guidance head is used only during training and removed for inference; the deployed model consists of a positional‑injection‑free ViT encoder and the task‑specific prediction module. Using DINOv3 ViT backbones, Guidance consistently improves performance on ImageNet‑100 classification, ADE20K semantic segmentation, and Hypersim monocular depth estimation, outperforming strong injected baselines such as learned absolute positional embeddings and rotary positional embeddings under identical training protocols. On ImageNet‑100, broader comparisons against representative injected positional designs further support Guidance's effectiveness. Guidance also improves robustness under resolution transfer, and multi‑resolution training further strengthens accuracy across input sizes. Overall, our results suggest that spatial inductive bias in ViTs need not be architecturally injected, but can be shaped through training‑time supervision. The code used for training and evaluation is publicly available in https://github.com/cloudlc/asg.
Authors:Jihyeok Jung, Jeewu Lee, Sanghyeop Kim, Chanhee Han, Seong Joon Oh
Abstract:
Existing egocentric benchmarks have primarily constructed the egocentric setting from first‑person‑view data, which makes it difficult to evaluate egocentric perspective itself in isolation. However, understanding first‑person‑view input and taking an egocentric perspective are separable abilities, especially when first‑person body cues are absent or when other agents are present. To isolate egocentric perspective understanding, we introduce EgoGapBench, a diagnostic benchmark for measuring action selection in multi‑agent egocentric scenes. We define the ability measured by this benchmark as Egocentric Action Selection (EAS): selecting an appropriate action from the agent's perspective in the presence of other agents. On EgoGapBench, humans answer reliably, whereas both open‑source and proprietary MLLMs perform substantially worse and systematically select actions performed by other visible agents. Fine‑tuning on existing egocentric data fails to close this gap and can even be detrimental. In contrast, fine‑tuning on EgoGapBench training data improves accuracy but does not reach human performance. These results show that EAS is difficult to acquire from first‑person‑view data alone, and that MLLMs should be evaluated and trained not only for scene understanding but also for egocentric action selection.
Authors:Prabod Rathnayaka, Fabian Waschkowski, Lukas Wesemann
Abstract:
We present BaseRT, a native Metal inference runtime for large language models (LLMs) on Apple Silicon, and report the highest inference throughput on this hardware to date. Existing runtimes, including llama.cpp and MLX‑based frameworks, incur overhead from abstractions not designed for Metal's execution model or Apple Silicon's unified memory topology. By building natively on Metal with chip‑specific kernel fusion, unified memory‑aware optimisation, and custom dispatch logic, BaseRT recovers performance that framework‑based approaches leave on the table. BaseRT supports a wide range of model families across eight quantisation formats (Q2 to FP16) on all Apple M‑series devices. In this paper, we evaluate the Qwen3, Llama 3.2, and Gemma 4 families at Q4 and Q8 quantisation on M3 and M4 Pro devices. BaseRT achieves up to 1.56x higher decode throughput than llama.cpp and up to 1.35x higher than MLX, with substantially larger margins on prefill for mixture‑of‑experts models, delivering consistent best‑in‑class throughput from sub‑1B to 30B parameter models. These results establish Apple Silicon as a more capable inference platform than previously reported, with direct implications for the emerging edge inference paradigm: as privacy requirements, latency constraints, and cloud cost pressures drive inference toward on‑device deployment, performance‑optimised local runtimes are a critical enabling layer for this transition. BaseRT is publicly available at https://github.com/basecompute/baseRT
Authors:Anindya Sarkar, Nasik Muhammad Nafi, Isaac Lyngaas, Muralikrishnan Gopalakrishnan Meena, Yevgeniy Vorobeychik
Abstract:
Diffusion models are highly effective at modeling complex data distributions, including images and text. However, in applications like personalized recommender systems, the objective often shifts to modeling specific regions of the distribution that maximize user preferences‑initially unknown but gradually uncovered through interactive feedback. This can naturally be framed as a reinforcement learning problem, where the goal is to fine‑tune a diffusion model to maximize a reward function based on preferences. However, the main challenge lies in learning a parameterized reward model, which typically requires large‑scale preference data‑something that is often not feasible in practice. In this work, we introduce Personalized Active Preference Alignment PAPA, a novel method that bypasses the requirement for a parametrized reward model by directly optimizing the diffusion model using real‑time user feedback. PAPA enables feedback‑efficient preference alignment, drawing inspiration from the variational inference framework. We demonstrate PAPA's effectiveness through extensive experiments and ablation studies across diverse class‑conditioned and fine‑grained alignment tasks. Additionally, based on theoretical insights, we propose an enhanced fine‑tuning strategy, referred to as EPAPA, that requires less computational budget and accelerates the fine‑tuning process, further boosting PAPA's suitability for real‑world deployment. Our code is made publicly available at https://github.com/NasikNafi/papa.
Authors:Junlong Liu, Haobo Wang, Weiqi Luo, Xiaojun Jia
Abstract:
Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs). While prior work has primarily studied attacks and defenses at the prompt level, we show that this prompt‑centric paradigm overlooks a structural vulnerability in stateful, function‑calling environments. In such applications, developer‑defined schemas, structured arguments, and untrusted tool outputs are interleaved into a single shared model context. This architecture expands the attack surface by blurring the boundary between trusted control logic and untrusted data, allowing adversarial intent to be distributed across a multi‑turn execution path. We exploit this architectural flaw through SMT, a black‑box attack framework based on Simulated Moderation Traces. Departing from purely prompt‑based interactions, SMT constructs a multi‑turn trajectory that simulates a legitimate moderation‑auditing workflow. Within this trajectory, a fabricated moderation frame leverages red‑team testing as a pretext to elicit harmful generations. The subsequent validation feedback treats safety refusals as execution failures, prompting refinements that gradually weaken the model's safety constraints and ultimately trigger harmful outputs. Extensive empirical evaluations on prominent commercial LLMs from five different providers across two standardized safety benchmarks show that SMT consistently achieves the highest average attack success rate and HarmScore while requiring a near‑minimal number of queries, substantially outperforming existing baselines. These findings demonstrate that prompt‑level sanitization alone is fundamentally insufficient for defending tool‑enabled LLM systems and highlight the urgent need for context‑aware validation across schemas, arguments, tool outputs, and accumulated conversation state. The code is available at https://github.com/liujlong27/SMT.
Authors:Yuan Qing, Chengzhi Mao, Boqing Gong
Abstract:
Large Vision‑Language Models (LVLMs) rely extensively on Visual Instruction Tuning (VIT) to elicit their multimodal reasoning capabilities. However, we find a discrepancy: VIT often packs multiple language tasks about the same image for conversational, multi‑turn training, whereas existing benchmarks evaluate LVLMs in isolated, single‑turn scenarios. The models can suffer from visual attention decay and contextual overfitting during multi‑turn training, making it hard for them to realize their full potential in the mismatched test phase. To close the gap, we propose learning with Stochastic Turn Depth (StochasT), which stochastically groups language tasks for the same image into clusters of varying sizes (turn depth) while preserving their organic order. Hence, while StochasT draws on Dropout and stochastic depth for ResNets, it does not actually drop anything to maximize the utility of the training data. Furthermore, we introduce a challenging, benchmark‑agnostic evaluation mechanism based on the Balanced Latin Square to measure LVLMs' robustness under varying contextual dependencies. Extensive experiments demonstrate that StochasT effectively grants LVLMs strong, harmonized capabilities for both single‑turn and multi‑turn use cases.
Authors:Yangfan Hu, Xuhan Tong, Haoyue Bai, Xi Ding, Shashank Muralidhar Bharadwaj, Siyang Cao, Robert Nowak, Jiawei Zhang
Abstract:
Large language models often produce hallucinated answers that violate prompt‑level constraints. A key diagnostic question is whether these failures reflect missing knowledge, or whether the model has the relevant information but follows the wrong inference path. We study this phenomenon as inference misalignment: a mismatch between the answer supported by the prompt and the answer favored by statistically salient latent associations. We formalize this view with a latent key‑task model, in which pretraining‑frequency imbalance can cause a shortcut path to dominate the constraint‑sensitive path and induce positive inference loss. The framework predicts two failure modes: task‑retrieval bias in entity disambiguation and key‑selection bias in action choice. We introduce TrapQA, a controlled diagnostic testbed with two components. ScientistQA tests disambiguation among similar scientists with supplementary factual probes, while Real‑Life Constrained QA tests everyday constraint following under salient shortcuts. Our results show that hallucination can arise from biased latent inference rather than absent knowledge alone.
Authors:Merve Atasever, Cagan Bakirci, Alfredo Reina Corona, Keyan Azbijari, Jyotirmoy V. Deshmukh
Abstract:
Reinforcement learning (RL) for quadruped locomotion commonly depends on fixed, hand‑crafted, and Markovian reward functions that limit both interpretability of learned policies and lack explicit control over gait behaviors. We introduce a framework where distinct gaits are specified using parameterized constraints expressed in Signal Temporal Logic (STL). These include safety bounds, gait synchronization constraints, command tracking, and actuation bounds. From these specifications, we develop a reward shaping mechanism that provides learning agents a dense, continuous reward landscape that encodes desired behavior. We define parametric STL templates for three speed regimes (walking‑trot, trot, bound), calibrate their parameters from reference rollouts, and compute rewards from using smooth approximations of STL robustness over the rollouts. The generated rewards can be used to provide shaped gradients compatible with Proximal Policy Optimization (PPO). We instantiate the approach on Google's Barkour quadruped robot in MuJoCo XLA (MJX). We use parallelization within the simulator to improve training speeds and use domain randomization to robustify learned policies. We show that compared to a baseline of hand‑crafted rewards, the STL‑shaped rewards yield tighter velocity tracking and more stable training. Videos can be found on our project website: https://stl‑locomotion.github.io/.
Authors:Ji Ha Jang, Hayeon Kim, Chulwon Lee, Junghun James Kim, Se Young Chun
Abstract:
CLIP (Contrastive Language‑Image Pre‑training) has become a de facto paradigm for image‑text alignment, but it struggles with long‑context descriptions (>77 tokens) due to absolute positional encoding and pretraining on short captions. In long contexts, sentences are often reordered, summarized, or partially omitted. Although prior works extend CLIP with longer positional encodings, they often suffer from degraded image‑text alignment under such text perturbations. We attribute this limitation to the Euclidean contrastive objective, which enforces strict one‑to‑one matching and lacks explicit mechanisms for modeling hierarchical relationships between global context and its constituent elements. To address this issue, we propose HyFL‑CLIP, a hyperbolic fine‑tuning framework that distills the well‑established text‑image alignment learned in Euclidean CLIP into hyperbolic space via cross‑manifold similarity distillation, leveraging its geometry to capture hierarchical and entailment relations. Our method models hierarchical semantics by linking summarized token‑wise features, long‑context descriptions, constituent short textual components, and images, capturing part‑whole relationships via hyperbolic entailment with Einstein midpoint aggregation. Experiments on diverse benchmarks, including long‑context cross‑modal retrieval, cross‑modal retrieval with caption perturbations, intra‑modality retrieval, and short‑text cross‑modal retrieval, show that HyFL‑CLIP achieves more robust long‑context understanding. In particular, it yields up to 19.5% improvement in long‑text cross‑modal retrieval under textual perturbations over the best prior method. We also show HyFL‑CLIP can be seamlessly integrated into other model frameworks by applying it to Stable Diffusion XL (SDXL).
Authors:Chanwoo Choi, Euntae Kim, Kyuho Lee, Youngsam Chun, Jinhee Jeong, Eunmi Kim, Myunggyo Oh, Junseo Jang, Buru Chang
Abstract:
Retrieval‑Augmented Generation (RAG) systems are vulnerable to poisoning attacks that inject malicious documents into the retrieval process to manipulate model outputs. Recent Agentic RAG systems are more robust to such attacks because they iteratively perform retrieval and reasoning, allowing them to ignore weakly relevant poisoned documents and preserve the reasoning chain induced by the user query. However, existing attacks on Agentic RAG systems often assume white‑box access to system prompts, reasoning traces, retrievers, or model parameters, limiting their applicability in realistic settings. In this paper, we study black‑box poisoning attacks against Agentic RAG systems, where the attacker can only publish externally retrievable poisoned documents. We propose KidnapRAG, a sequential poisoning attack that hijacks the agent's multi‑step reasoning chain using three role‑specific documents: Bait, Chain‑Link, and Mal‑Ins, which attract initial retrieval, induce query reformulation, and provide attacker‑controlled evidence, respectively. Experiments across multiple Agentic RAG frameworks, LLM backbones, and benchmarks show that KidnapRAG consistently outperforms existing poisoning baselines under black‑box conditions. Further analyses show that KidnapRAG progressively weakens the original retrieval intent, redirects retrieval behavior, and increases reliance on attacker‑controlled evidence. Our code is publicly available at https://github.com/chanwoochoi316/KidnapRAG.
Authors:Wei Sun, Weixia Zhang, Hongjian Zhan, Mingkai Lu, Yixuan Gao, Guangtao Zhai
Abstract:
We present DroneIQA‑VLE, our solution to the ICME 2026 Drone‑IQA Grand Challenge on Target‑aware Image Quality Assessment for Low‑altitude UAV Images. The framework jointly predicts global, target, and background quality scores by ensembling two complementary pipelines: (1) SigLIP2 vision encoders with multi‑task regression heads, and (2) a LoRA‑adapted Qwen3.5‑9B multimodal large language model for quality score regression. The final global quality prediction is obtained by arithmetically averaging the outputs of both pipelines. Our method achieves 2nd place in the challenge, demonstrating its effectiveness. The code is available at https://github.com/sunwei925/DroneIQA‑VLE.
Authors:Saad Wazir, Patrick Dominique Vibild, Dinh Phu Tran, Seongah Kim, Daeyoung Kim
Abstract:
Medical image segmentation relies on the ability of encoder‑decoder architectures to translate rich feature representations into accurate pixel‑level predictions under challenging conditions such as low contrast, structural ambiguity, and scale variability. While recent advances in large‑scale pretraining and transformer‑based encoders have substantially improved feature extraction, segmentation accuracy remains constrained by decoder design, particularly in terms of cross‑scale alignment, contextual integration, and boundary preservation. In this work, we revisit medical image segmentation from a decoder‑centric perspective and propose a context‑aware gated decoder that systematically regulates feature fusion and contextual aggregation throughout the decoding process. The proposed decoder integrates lightweight multi‑scale channel recalibration, gated skip fusion with spatial competition and a global context aggregation mechanism that injects encoder‑wide information into intermediate decoding stages. This design enables effective translation of strong pretrained encoder representations into spatially consistent predictions. Extensive experiments across 11 medical image segmentation benchmarks validate the effectiveness and demonstrate that the proposed approach consistently outperforms strong baselines while remaining computationally practical. Code: https://github.com/saadwazir/MedCAGD
Authors:Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, Mubarak Shah
Abstract:
Safety alignment of text‑to‑image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts. Recent methods often appear to deliver high safety with high utility, but this conclusion rests largely on coarse global utility metrics (e.g., FID, CLIPScore) that are insensitive to fine‑grained semantic correctness, creating an illusion of high utility. We show that when utility is measured with structured evaluation, this illusion breaks: on TIFA (Text‑to‑Image Faithfulness evaluation with Question Answering), safety‑aligned models suffer substantial drops in semantic fidelity, including failures in object counts, attributes, and relationships. To diagnose the source of this gap, we analyze the text‑encoder prompt embedding space and uncover semantic collapse, a contraction of embedding spread coupled with distortion of inter‑prompt similarity structure, which strongly correlates with structured utility loss. Guided by this insight, we propose StructureAware Geometric Regularization (SAGE), a safety alignment objective that explicitly preserves embedding spread and inter‑prompt relational structure during adaptation. Our method restores structured utility (TIFA +5.0% over prior state‑of‑the‑art) while maintaining strong safety performance and competitive coarse‑grained utility scores. Our source code and trained models are available at https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/.
Authors:Jongchan Park, Seungjun Oh, Seungho Baek, Yusung Kim
Abstract:
Unsupervised Reinforcement Learning (URL) aims to pre‑train scalable, skill‑conditioned policies without extrinsic rewards, serving as a foundation for downstream control tasks. Despite recent progress, we argue that current off‑policy URL methods are limited by two critical, overlooked bottlenecks: (1) non‑stationary skill semantics and (2) brittle generalization. To address these challenges, we propose GenDa (Generalizable Data‑efficient Agent), a unified framework for robust unsupervised reinforcement learning. First, we introduce a skill relabeling mechanism to mitigate non‑stationarity and significantly improve data efficiency for pre‑training. Second, we propose a Complementary Information Bottleneck (CIB), encouraging the learned skill policy to focus on ego‑centric features and become robust to distribution shifts for downstream tasks. Through various experiments, we demonstrate that GenDa significantly enhances the scalability of URL with superior generalizability and data efficiency. Our code and videos are available at https://ihatebroccoli.github.io/official‑GenDa.
Authors:Jing Gao, Wei Wang, Feiran Wang, Yan Yan
Abstract:
We present LIST3R, an instance‑aware framework for long‑sequence 3D reconstruction inspired by the way humans organize spatial memory around stable and recognizable objects. LIST3R organizes long‑sequence reconstruction around instance anchors, using them to reconnect fragmented subsequences and consolidate local observations into a coherent global 3D scene. Given a long video, our approach partitions it into overlapping subsequences and builds a structured local instance library for each partial reconstruction, maintaining persistent trackable anchors with semantic and geometric evidence. These anchors are matched across subsequences to recover revisited regions and provide object‑aware constraints for fragment alignment, producing a consistent global reconstruction. During this process, the evolving geometric evidence updates the local instance libraries and progressively organizes them into a unified global 3D instance library. Experiments on long‑sequence benchmarks show that our method produces more accurate trajectories and higher‑quality 3D reconstructions, highlighting the effectiveness of persistent instance anchors for organizing long‑horizon 3D reconstruction. Our code is available on the project page: https://yixn965.github.io/LIST3R/.
Authors:Siyuan Yao, Ziqi Wang, Ruiqi Yu, Junqi Huang, Wenqi Ren, Xiaochun Cao
Abstract:
Domain adaptive visual object tracking under adverse weather conditions has garnered significant attention in recent years. Despite the impressive performance, existing methods heavily rely on the large‑scale video frames from both source and target domains, which is impractical under rigid resource constraints where source data is unavailable. To overcome this limitation, we propose SFDATrack, a generalized source‑free domain adaptive tracker that merely leverages adverse weather samples from the target domain for robust state estimation. Specifically, SFDATrack first employs a mean‑teacher backbone with Dual Interactive Mamba (DIM) blocks to distill the candidate target tokens that are resilient to weather variations from classified, augmented samples. Afterwards, we introduce a hyperspherical prototype projection (HPP) module to project these tokens onto multi‑domain prototypes within a latent hyperspherical space. By enforcing both domain‑specific and domain‑invariant properties of the multi‑domain prototypes, SFDATrack can be seamlessly adapted to diverse weather conditions with powerful generalizability. Extensive experiments evaluated on various benchmarks demonstrate that SFDATrack achieves superior performance compared to state‑of‑the‑art approaches. The code is available at https://github.com/watcherBR0/sfdatrack.
Authors:Yujin Huang, Xin Zheng, Xingliang Yuan, Kwok-Yan Lam
Abstract:
Mobile on‑device AI (MoAI) systems that integrate locally deployed AI models with conventional mobile software components are emerging as a key paradigm for delivering intelligent functionality directly on end‑user devices. By moving inference from remote cloud services to the local mobile environment, such systems enable privacy‑preserving, low‑latency, and offline‑capable AI functionality, yet introduce new security risks arising from the local storage of AI models. This paper presents the first comprehensive systematization of knowledge on MoAI security, covering security pillars, attack landscape, and defense landscape of MoAI systems. We further identify unresolved gaps in current attack and defense research and point to promising directions for future research in this emerging area. Our work establishes the first systematic framework for understanding the attack and defense landscapes of MoAI systems, serving as a foundation for building secure MoAI systems and advancing research in this critical domain. Companion resources are available at https://github.com/Jinxhy/Awesome‑MoAI‑Security.
Authors:Prabal Gupta
Abstract:
We present a real‑time musical interface that converts natural‑language scene descriptions into evolving procedural soundscapes. A performer types a prompt such as "warm jazz cafe at midnight" and steers it through direct parameter adjustments ‑ stepping brightness down, switching a rhythm style ‑ each producing a predictable, audible shift without re‑prompting. Where GPU‑bound text‑to‑audio systems synthesize monolithic waveforms, our instrument generates human‑readable configurations over a categorical schema, enabling fine‑grained performer control; most valid combinations are designed to sound musically coherent. Three interchangeable backends ‑ embedding retrieval for sub‑second CPU‑only use, hosted LLMs via API, and a fine‑tuned 270M local model ‑ all emit the same schema. A live generator architecture continuously emits audio while resolving new instructions in the background, crossfading seamlessly when ready; even when an LLM takes 5‑12 seconds to respond, the audience hears uninterrupted sound ‑ reframing text‑to‑music as an ongoing performable stream rather than a one‑shot generation. We evaluate text‑audio semantic alignment using LAION‑CLAP on held‑out prompts as a technical proxy, finding that retrieval‑based configuration outperforms random valid configurations on this metric, while noting that LAION‑CLAP also informed retrieval‑map construction. We report performance observations, informal listener feedback, and release materials for the SDK, dataset artifacts, model, and audiovisual performance interface.
Authors:Xiangyue Liu, Zijian Zhang, Miles Yang, Zhao Zhong, Liefeng Bo, Ping Tan
Abstract:
Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge. However, accommodating continuous generative objectives alongside discrete understanding tasks causes severe gradient conflicts. Existing architectures, including standard Mixture‑of‑Experts (MoE), are highly susceptible to representation overwriting. Even structurally partitioned paradigms like Mixture‑of‑Transformers (MoT) remain vulnerable to catastrophic forgetting, severely impeding multimodal scalability. In this work, we introduce Rosetta, a composable native multimodal pretraining framework designed for seamless and non‑destructive modality expansion. Rosetta adopts a modular paradigm where core foundational knowledge is preserved within global shared experts, while modality‑specific capabilities are distributed across plug‑and‑play experts. To guarantee non‑destructive composition, we propose Momentum‑Anchored Orthogonal Projection (MAOP). MAOP leverages the optimizer's momentum state as an implicit semantic anchor, selectively neutralizing conflicting gradient components from new modalities while preserving synergistic updates. Extensive evaluations demonstrate that, while standard MoE and MoT architectures suffer catastrophic forgetting of previously acquired knowledge, Rosetta robustly preserves established language and visual understanding. Furthermore, it delivers superior image generation and unlocks cross‑modal synergy, paving the way for truly composable and unified multimodal foundation models. To facilitate further multimodal research, we release our code and checkpoints to the community. Project page at https://rosetta‑lmm.github.io/.
Authors:Thinh Phan, Hao Vo, Khoa Vo, Thanh Ngo, Cuong Pham, Ngan Le
Abstract:
The core challenge in multi‑view pedestrian detection (MVPD) lies in effective aggregation of visual features from different viewpoints for robust occlusion reasoning. Recent approaches have addressed this by first projecting image‑view features onto a Bird's Eye View (BEV) map, where ground localization is then performed. Despite impressive performance, the perspective transformation induces severe distortion, causing spatial structure break and degrading the quality of object feature extraction. The blurred and ambiguous features hinder accurate BEV point localization, especially in densely populated regions. Moreover, the strong mutual relationship between the BEV ground point and image bounding boxes is not capitalized on. Although multi‑view consistency of 2D detections can serve as a powerful constraint in BEV space, these detections are commonly treated as auxiliary signals rather than being jointly optimized with the primary task.In this work, we propose MVDGC, a unified framework that \emphjointly estimates pedestrian locations on the BEV plane and 2D bounding boxes in image views. MVDGC employs a \emphsparse set of 3D cylindrical queries that embraces geometric context across both BEV and image views, enforcing dual spatial constraints for precise localization. Specifically, the geometric constraints is established by modeling each pedestrian as a vertical cylinder whose center lies on the BEV plane and whose projection casts a rectangular box in the image views. These queries function as shape anchors that directly extract 2D features from the intact image‑view features using camera projection, eliminating projection‑induced distortions. The 3D cylindrical query enables the unification of BEV and ImV localization into a single task: 3D cylinder position and shape refinement. Code is available at: https://github.com/UARK‑AICV/MVDGC
Authors:Edward Y. Chang, Longling Geng, Emily J. Chang
Abstract:
LLMs, solvers, and agent teams increasingly generate workflow actions, repairs, and plans, but a generated action may be syntactically valid yet stale, infeasible, conflicting, or destructive of the evidence that triggered a repair. We introduce Agentic Transaction Processing (ATP), a transaction model that treats generated actions as untrusted proposals until they pass deterministic admission under a declared, executable constraint set C. The principle is two‑sided: a proposal is not truth, and no proposal foresees every disruption: anything may propose, but only the runtime admits and commits, and when an unforeseen disruption strikes it repairs reactively within bounds rather than trusting a fresh proposal. Relative to C, committed‑state correctness becomes independent of the competence, honesty, or learning of the proposing layer. We realize ATP in Mnemosyne, a runtime with an append‑only transition log, effective‑state projection, dependency‑safe compensation, and active commitment records, and prove four safety properties relative to C (authority separation, serial‑equivalent generative admission, evidence‑preserving repair, and obligation containment) together with a bounded‑reactive‑repair guarantee for its localized repair protocol (LCRP). A reproducible artifact rejects the targeted violations across nine falsification tests while still admitting valid work, at under 6% projection‑and‑validation overhead, and bounded local repair edits an order of magnitude fewer operations than global recompute. Mnemosyne is open source: https://github.com/eyuchang/Mnemosyne/tree/arxiv‑atp‑rq1‑rq9b‑r8‑v2.
Authors:Xin Li, Wenhui Zhu, Xuanzhao Dong, Xiwen Chen, Yanxi Chen, Yujian Xiong, Hao Wang, Oana M. Dumitrascu, Yalin Wang
Abstract:
Medical image segmentation is dominated by U‑Net‑style encoder‑decoder architectures. Vision Transformers (ViTs) overcome the limited receptive field of convolutional networks through self‑attention, enabling modeling of long‑range dependencies. Early ViT‑based segmentation methods typically retained U‑Net‑style decoders because pretrained ViT representations were insufficient to support accurate dense prediction. Recent advances in large‑scale pretraining have redefined the representation capability of ViTs, reducing the reliance on U‑Net‑style decoder architectures in modern vision models. This prompts two questions: Is the U‑Net paradigm still necessary for medical image segmentation? If not, how should an encoder‑only segmentation framework be designed? Motivated by these questions, we explore key architectural choices for encoder‑only medical image segmentation based on modern ViT backbones and establish a query‑based encoder‑only design with multi‑level query modeling and learnable block fusion, realized in Encoder‑only Segmentation (EoSeg). Extensive experiments across seven benchmark datasets spanning CT, MRI, histopathology, endoscopy, and dermoscopy validate the effectiveness of the proposed design across diverse medical imaging modalities, including mDice scores of 85.50% on Synapse, 91.73% on ACDC, and 93.27% on GlaS. The results demonstrate that a U‑Net‑style decoder is no longer necessary for medical image segmentation with modern ViT backbones and further show that EoSeg provides an effective encoder‑only design. Code is available at: https://github.com/Retinal‑Research/EoSeg
Authors:Ruikang Zhao, Zhenting Wang, Han Gao, Ligong Han
Abstract:
Reinforcement learning for diffusion large language models (dLLMs) has largely moved to trajectory‑aware methods. The current state of the art, TraceRL, holds that random masking is mismatched with the model's inference trajectory, and it reconstructs that trajectory during training by slicing each rollout into up to K/s trajectory‑aligned training samples, a cost that grows with the block size K. We show that this mismatch can be mitigated without reconstructing the trajectory. Our method, SLIM‑RL, bounds the commit risk of each rollout step with a tau‑budget decoder, reducing aggregate commit risk in the training data. During optimization, SLIM‑RL trains on these risk‑controlled rollouts with a trace‑free random‑masking objective that adapts variance‑reduction tools, combining sequence‑level importance sampling, deterministic quadrature over masking levels under a mean‑preserving, monotonically decreasing per‑block mask schedule that we introduce. On SDAR‑4B, SLIM‑RL matches TraceRL's best MATH500 accuracy on only 0.46x its training samples at block size 16, improving over TraceRL by 6.32% on MATH500 and 11.05% on GSM8K under matched dynamic sampling. At block size 4, the 4B SLIM‑RL surpasses the larger LLaDA‑8B and Dream‑7B dLLMs on math, exceeding LLaDA‑8B by 10.76% on MATH500 while staying below the autoregressive Qwen2.5‑7B. On code, it improves over TraceRL by 4.20% on MBPP and 3.65% on HumanEval. The tau‑budget decoder transfers training‑free across LLaDA, Dream, and SDAR. The source code is available at https://github.com/laolaorkkkkk/SLIM‑RL .
Authors:Bharat Srikishan, Javier E. Santos, Nikhil Muralidhar, Charles D. Young
Abstract:
Many scientific systems exhibit uncertainty from stochastic forcing, unresolved degrees of freedom, or imperfect observations, making reliable surrogate forecasting fundamentally distributional rather than pointwise. For such systems, deterministic neural surrogates fail to capture statistical measures and forecast uncertainty. We introduce TRIE, an evaluation framework for stochastic PDE surrogates that asks whether models reproduce invariant measures, provide trustworthy predictive uncertainty, and scale to efficient probabilistic generation. We demonstrate TRIE on two stationary chaotic spatially extended SPDEs, stochastic Kuramoto‑‑Sivashinsky and stochastic Kolmogorov flow, across 11 parameter values. Our evaluation shows that standard pointwise‑trained neural surrogates can produce plausible short rollouts while failing to match long‑time statistical structure. Approximate uncertainty methods such as Monte Carlo dropout and heteroscedastic Gaussian likelihoods produce stochastic forecasts, but are often miscalibrated and overconfident under temporal and spatial uncertainty diagnostics. Across these criteria, generative models provide the most consistent performance, accurately capturing invariant measure statistics and achieving the lowest CRPS in all reported probabilistic settings. Finally, we show that latent generative models with automatic dimension discovery retain much of this statistical fidelity while reducing Kolmogorov inference time by roughly 12×. We release our code and data at https://github.com/scailab/TRIE‑SPDE‑Bench to support reproducible evaluation of stochastic PDE forecasting models.
Authors:Luke Chen, Cheng-Ju Wu, David R. Martin, Qilin Ye, Pramod Khargonekar, Mohammad Abdullah Al Faruque
Abstract:
Collaborative‑perception enables multi‑robot systems to enhance situational awareness by sharing perceptual information. Existing collaborative‑perception systems face an inherent trade‑off between communication bandwidth requirements and perception accuracy, where methods that exchange more information achieve better perception results at the cost of increased communication overhead. However, real‑world communication networks impose bandwidth constraints that require minimizing communication overhead without sacrificing perception performance. To address this challenge, we propose HydraCollab, an adaptive collaborative‑perception framework that (i) selectively transmits the most informative sensor features and (ii) dynamically employs collaboration strategies (intermediate or late) based on spatial confidence maps. Extensive evaluations on the V2X‑R, V2X‑Radar and UAV3D‑mini datasets demonstrate that HydraCollab achieves the best overall trade‑off between accuracy and communication cost among existing collaborative‑perception methods. Relative to SOTA Where2comm, HydraCollab uses only 41% of the bandwidth on V2X‑R and 26% on V2X‑Radar while improving performance by 0.78% and 0.75% respectively. Our code and models are available at https://github.com/AICPS/HydraCollab.
Authors:Andrianos Michail, Stylianos Psychias, Michelle Wastl, Simon Clematide, Rico Sennrich, Juri Opitz
Abstract:
Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover only a limited set of languages, are often domain‑specific, susceptible to overfitting, and poorly representative of low‑resource languages. To address these limitations, we introduce ALEE, a framework that extends Sentence Smith (Li et al., 2025) to the cross‑lingual and paragraph level. ALEE uses Abstract Meaning Representations (AMR) to generate English minimal pairs with controlled, fine‑grained semantic shifts, which are paired with translations in target languages. This approach enables targeted diagnostics for models in any language with English parallel data. We conduct a large‑scale empirical study across a diverse set of embedding models and 275+ languages spanning three parallel datasets. On ALEE, performance varies substantially across languages, text lengths, and linguistic phenomena, exposing persistent gaps in cross‑lingual semantic representation that track language prevalence in training resources and subword tokenization. We release ALEE at https://github.com/Andrian0s/any‑lang‑embed‑eval
Authors:Qian Ma, S M Rayeed, Charles V. Stewart, Qiong Wu, Yao Ma
Abstract:
Knowledge‑Based Visual Question Answering (KB‑VQA) aims to evaluate whether Visual Language Models (VLMs) can retrieve, ground, and reason over external structured knowledge beyond visual evidence. In practice, answer accuracy is widely adopted as the primary evaluation metric, implicitly treating correctness as a proxy for knowledge‑grounded reasoning. However, for existing KB‑VQA benchmarks, this proxy relies on critical assumptions that are often overlooked and rendered unreliable by benchmark issues: annotated answer must be derivable from the associated knowledge base, question must be well‑posed with sufficient constraints, and visual setting must meaningfully require grounded disambiguation. In this work, we show that these assumptions are systematically violated in existing KB‑VQA benchmarks. Our audit reveals substantial instances with missing or contradicted answers and underspecified questions that render accuracy a misleading metric. Furthermore, we find that existing datasets rely on visually trivial, single‑entity scenes that bypass the need for sophisticated visual‑to‑knowledge mapping. We demonstrate that even with controlled architectures, these flaws lead to distorted model rankings and overestimations of reasoning capabilities. To address this, we introduce (1) a principled audit‑and‑repair protocol that restores answer derivability and question clarity, and (2) a controlled multi‑entity augmentation protocol that introduces visual ambiguity to challenge initial retrieval and grounded reasoning. Re‑evaluation under corrected and augmented settings yields markedly different performance trends. Our findings call for rethinking evaluation protocols and designing more interaction‑aware KB‑VQA benchmarks that prioritize verifiable reasoning over simple matching.
Authors:Yong Yi Bay, Kathleen A. Yearick
Abstract:
Three of the most popular methods for training language models to reason look like three different tricks. They are not. All three adjust a single number: standard deviation, reflecting how much a prompt's sampled answers disagree. When such a model is trained, it answers each problem many times, and an automatic checker marks every answer right or wrong. The standard deviation of those marks measures the disagreement: largest when the answers split evenly between right and wrong, and zero when they all agree. Group Relative Policy Optimization (GRPO) divides by this number, GRPO Done Right (Dr. GRPO) drops the division, and Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO) discards the groups where it is zero. Each is presented as its own fix, yet this paper proves they are three settings of one dial. That dial is not cosmetic: for right‑or‑wrong rewards, the disagreement is exactly the size of the training update, the group‑standard‑deviation identity. A split group teaches the most, while a unanimous group teaches nothing and falls silent. The same result says which problems deserve the most weight and how many tries each one needs. This paper confirms the intuition on a large real difficulty dataset (Big‑Math) and in a controlled training run. What looks like a harmless normalization step is the dial that decides where learning happens and how strongly.
Authors:Yunhan Wang, Eshika Khandelwal, Edson Araujo, Walid Bousselham, Nina Shvetsova, Hilde Kuehne
Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated remarkable abilities when analyzing images, yet translating these capabilities to few‑shot image classification remains challenging. To bridge this gap, we present DeCoDe, a simple yet effective technique that enables off‑the‑shelf MLLMs to act as strong few‑shot classifiers without any additional training. Our approach builds on the idea of few‑shot classification as a set of pairwise image comparisons, decomposing the task into a set of binary decisions. Given a query image and a support image from a candidate class, the MLLM is prompted to decide whether the two images depict the same class. The logit corresponding to an affirmative response is then used as a similarity score to assign the query image to the most likely class. While this already yields good results, we show that providing additional high‑level information, such as the data domain, to the model further improves performance. Our evaluation provides an extensive analysis of various inference variants on a suite of twelve datasets, six established and six newly curated few‑shot benchmarks spanning across diverse domains. The results show that the proposed simple decomposition technique can turn off‑the‑shelf MLLMs into powerful few‑shot learners, significantly outperforming current state‑of‑the‑art few‑shot methods on both standard and novel domains. Code is available at https://github.com/yunhanwang1105/DeCoDe.
Authors:Asif Mahbub, Md. Abir Hossain, Nabil Bin Hannan
Abstract:
Domain Name System (DNS) resolution in Internet of Things (IoT) networks presents unique challenges due to resource constraints, unreliable connectivity, and security vulnerabilities. Traditional centralized DNS architectures introduce single points of failure. This paper presents MeshDNS, a cooperative DNS resolution framework designed for resource‑constrained IoT environments operating under shared‑key admission. MeshDNS employs a decentralized architecture where nodes maintain cache awareness using hash‑based summaries and secure cold‑cache misses via Ed25519‑signed, identical‑answer quorum voting. Our implementation on commodity ESP8266 microcontrollers (sub‑50 KB usable RAM, 80 MHz) achieves a 0.47 ms warm‑cache resolution, outperforming native mDNS baselines (1.39 ms). To secure initial cold‑cache misses, MeshDNS trades a predictable ~1.3‑1.7s cryptographic penalty to successfully isolate Byzantine faults among admitted peers. Assuming a threat model where physical hardware extraction remains out of scope, MeshDNS demonstrates Byzantine fault isolation. We validated the framework via a 5‑node physical testbed and discrete‑event simulations scaling to 1,000 nodes; the results demonstrate that MeshDNS maintains resilient local name caches for persistent edge telemetry under network churn. Code is available at https://github.com/mahbubasif/MeshDNS‑Artifact.
Authors:Hussein Chouman, Wataru Sasaki, Tomokazu Matsui, Hirohiko Suwa, Keiichi Yasumoto
Abstract:
Mechanistic interpretability has produced a rich inventory of component‑level analyses that characterise what neural‑network components encode and how they interact. Their outputs, however, are not easily reusable: selectivity tables, circuit diagrams, and feature lists remain locked in per‑study notebooks ‑ non‑composable, not queryable in natural language, and not directly actionable for downstream audit or intervention. We study the representation layer that sits between these analyses and downstream use as a bottleneck that can be evaluated independently, and introduce Manifestation Units, a typed tuple protocol (E, S, R, D, G) extended with attention‑head primitives (T) for transformer architectures, organising per‑component statistics into structured fields populated automatically and queried through hybrid retrieval. Instantiated across generative vision (beta‑VAE), discriminative vision (CNN), and language (GPT‑2), the protocol supports two findings: typed structure substantially outperforms unstructured baselines on retrieval, and CNN filters retrieved by the schema satisfy causal sufficiency and necessity criteria under matched‑budget controls. The schema absorbs attention‑head primitives without modification, set‑recovers known IOI circuit members under retrieval‑budget‑matched controls, and reveals an irreducible two‑field core (S+R) with remaining fields either redundant or actively interfering. We present this as schema infrastructure for mechanistic interpretability rather than frontier‑scale validation.
Authors:Jacky H. T. Yip, Alessandro Mininno, Gary Shiu
Abstract:
We propose a Transformer‑based Reinforcement Learning architecture, "LB‑Explorer", to search for heterotic line bundle standard models arising from compactifications on smooth Calabi‑Yau (CY) threefolds. We focus on E_8× E_8 heterotic string theory compactifications on CY with abelian line bundles to produce \textSU(5)× \textS(\textU(1)^5) symmetry, whose \textSU(5) can be further broken to an MSSM‑like gauge group using appropriate discrete Wilson lines. We test the LB‑Explorer environment on complete intersection Calabi‑Yau (CICY) manifolds, though the neural network architecture naturally generalizes to any CY admitting a simplicial Mori cone and a freely‑acting discrete symmetry. The LB‑Explorer efficiently learns constraints on the line bundle sums, guaranteeing the E_8 gauge embedding, anomaly cancellation, poly‑stability (supersymmetry), chirality of the spectrum, and the absence of exotic matter. Valid configurations can be subsequently filtered by imposing the missing constraints, such as the equivariant structure of the line bundle sum and further requirements on the particle spectrum. In this direction, we introduce a hybrid architecture incorporating CP‑SAT solvers that aims to impose some of the conditions exactly by perturbing solutions found by the LB‑Explorer. The versatility and scalability of the LB‑Explorer make it a powerful tool for navigating the string landscape with a large number of moduli. The code and tools necessary to reproduce our findings are available at https://github.com/alexmininno/LB‑Explorer
Authors:Rui Hao, Qiankun Li, Junyuan Mao, Linghao Meng, Dirui Xie, Dayu Tan, Zhigang Zeng
Abstract:
Multimodal large language models (MLLMs) show strong promise for clinical VQA and radiology report generation, yet inference‑time hallucinations still undermine trustworthy use: models can produce fluent conclusions that conflict with imaging evidence. Existing mitigation strategies typically rely on additional training, external retrieval/knowledge bases, or multi‑stage post‑hoc verification, which increases cost and pipeline complexity and often generalizes poorly across models and tasks.To address this, we propose a holistic, training‑free evidence‑injection framework that systematically mitigates hallucinations through dual‑side evidence injection. By leveraging ROI priors acquired using MedSAM in our implementation, we recalibrate the visual perception trajectory via ROI‑guided activation modulation while anchoring the textual reasoning trajectory by mapping anatomical coordinates into discrete semantic tokens as verifiable external memory. Then we introduce a task‑aware dynamic router to select modality‑specific interventions based on task semantics, balancing perceptual grounding and linguistic fluency. We conduct systematic evaluations on 2 tasks and 5 datasets using \textttLLaVA‑1.5‑7B, \textttLLaVA‑Med‑1.5‑7B, \textttQwen3‑VL‑8B/32B, and \textttInternVL‑3.5‑8B/38B. Controlled ablations and visualizations further validate the framework, which consistently outperforms baselines across medical benchmarks, improving close‑ended accuracy by up to ~\mathbf6%\uparrow and reducing open‑ended hallucinations by ~\mathbf35%\downarrow. The code has been made available on GitHub: \hrefhttps://github.com/Henry991115/SPRG\textcolorbluehttps://github.com/Henry991115/SPRG.
Authors:Ying Chen, Jinyue Li, Qiankun Li
Abstract:
Image quality is critical for accurate medical diagnosis. However, MRI, CT, and ultrasound images are often of low resolution and quality due to cost constraints, complicating the visualization of key anatomical structures and lesions. While such limitations are common in practice, traditional methods treat image enhancement as a separate preprocessing step, failing to fully leverage its potential synergy with image segmentation. To address this, we propose DiSIINet (Diffusion‑based Symbiotic Information Interaction Network), which is built on the principle that enhancement and segmentation should mutually reinforce each other in a unified model. Based on Denoising Diffusion Implicit Models (DDIM), DiSIINet integrates an enhancement branch and a segmentation branch. These branches interact through a novel Symbiotic Information Interaction (SII) module, which facilitates dynamic, feature‑level information exchange via cross‑attention during the reverse diffusion process. This design enables both tasks to iteratively improve each other. The DDIM backbone ensures high‑quality output and efficient inference through deterministic sampling. Experiments on multi‑modal medical datasets (MRI, CT, ultrasound) show that DiSIINet achieves significant performance improvements compared to sequential or independent enhancement and segmentation approaches. The code is available at: https://github.com/Reconsider80/DiSIINet.
Authors:Xuan Zhao, Andy Chiu, Gengyu Wang
Abstract:
Information localization within massive repositories is a cornerstone of agentic LLM systems. While synthetic data‑driven optimization has proven successful in training LLMs, little attention has been paid to optimizing the agent's working environment (the repository itself) in a data‑driven manner. To bridge this gap, we present Libra, a self‑evolving framework that introduces mutable "catalogs" (hierarchical Markdown files serving as navigable indices) into the repository. Libra runs an LLM‑driven optimization loop where a Prompter generates synthetic queries, a frozen Solver attempts to resolve them by navigating the catalogs, and a Healer rewrites the catalogs in response to the Solver's localization failures. Evaluations across 12 SWE‑bench Lite repositories demonstrate that this environmental healing yields continual, logarithmic improvements in code localization accuracy. Furthermore, these environmental improvements transfer zero‑shot across different LLMs and problem sets. Although the focus of this paper is to study the general behavior of such a system, we also demonstrate that a minimalist coding agent equipped with Libra‑optimized catalogs outperforms state‑of‑the‑art baselines. Code is available at https://github.com/salesforce‑misc/Libra and data at https://huggingface.co/datasets/Salesforce/Libra.